Running Dataframe_image package on python anywhere DOES NOT WORK : Forums : PythonAnywhere

Running Dataframe_image package on python anywhere DOES NOT WORK

Hi,

I'm trying to get dataframe_image package to run on pythonanywhere (on jupyter notebook and python script) to convert pandas dataframe into an image, but it's not working and i'm not able to figure out what teaks to get it running. the code works perfectly fine on my laptop

my code:

import pandas as pd. 
import dataframe_image as dfi.

d = pd.DataFrame({'one':[1,1,1,1], 'two':[2,2,2,2]}).   
dfi.export(d, 'df_image.png', fontsize=32, max_rows=-1)

here is the error I get:

[1228/175855.081738:ERROR:platform_thread_posix.cc(135)] pthread_create: Resource temporarily unavailable (11)
[1228/175855.082516:FATAL:thread_pool_impl.cc(159)] Check failed: service_thread_.StartWithOptions(service_thread_options). 
#0 0x55fcfd8ea6b9 (/usr/lib/chromium/chromium+0x5b4b6b8)
#1 0x55fcfd856923 (/usr/lib/chromium/chromium+0x5ab7922)
#2 0x55fcfd869f92 (/usr/lib/chromium/chromium+0x5acaf91)
#3 0x55fcfd86a9be (/usr/lib/chromium/chromium+0x5acb9bd)
#4 0x55fcfd8c491f (/usr/lib/chromium/chromium+0x5b2591e)
#5 0x55fcfbe9166a (/usr/lib/chromium/chromium+0x40f2669)
#6 0x55fcfd82fdb6 (/usr/lib/chromium/chromium+0x5a90db5)
#7 0x55fcfd82fc2c (/usr/lib/chromium/chromium+0x5a90c2b)
#8 0x55fcfd82d5c4 (/usr/lib/chromium/chromium+0x5a8e5c3)
#9 0x55fcfd82de9e (/usr/lib/chromium/chromium+0x5a8ee9d)
#10 0x55fcfd8501d4 (/usr/lib/chromium/chromium+0x5ab11d3)
#11 0x55fcfd84ff03 (/usr/lib/chromium/chromium+0x5ab0f02)
#12 0x55fcfac0018c ChromeMain
#13 0x7f54beee2083 __libc_start_main
#14 0x55fcfac0002a _start

Received signal 6
#0 0x55fcfd8ea6b9 (/usr/lib/chromium/chromium+0x5b4b6b8)
#1 0x55fcfd856923 (/usr/lib/chromium/chromium+0x5ab7922)
#2 0x55fcfd8ea241 (/usr/lib/chromium/chromium+0x5b4b240)
#3 0x7f54c364d420 (/usr/lib/x86_64-linux-gnu/libpthread-2.31.so+0x1441f)
#4 0x7f54bef0100b gsignal
#5 0x7f54beee0859 abort
#6 0x55fcfd8e90a5 (/usr/lib/chromium/chromium+0x5b4a0a4)
#7 0x55fcfd86a3a3 (/usr/lib/chromium/chromium+0x5acb3a2)
#8 0x55fcfd86a9be (/usr/lib/chromium/chromium+0x5acb9bd)
#9 0x55fcfd8c491f (/usr/lib/chromium/chromium+0x5b2591e)
#10 0x55fcfbe9166a (/usr/lib/chromium/chromium+0x40f2669)
#11 0x55fcfd82fdb6 (/usr/lib/chromium/chromium+0x5a90db5)
#12 0x55fcfd82fc2c (/usr/lib/chromium/chromium+0x5a90c2b)
#13 0x55fcfd82d5c4 (/usr/lib/chromium/chromium+0x5a8e5c3)
#14 0x55fcfd82de9e (/usr/lib/chromium/chromium+0x5a8ee9d)
#15 0x55fcfd8501d4 (/usr/lib/chromium/chromium+0x5ab11d3)
#16 0x55fcfd84ff03 (/usr/lib/chromium/chromium+0x5ab0f02)
#17 0x55fcfac0018c ChromeMain
#18 0x7f54beee2083 __libc_start_main
#19 0x55fcfac0002a _start
  r8: 0000000000000000  r9: 00007ffd80a084d0 r10: 0000000000000008 r11: 0000000000000246
 r12: 000055fd03843020 r13: 000055fd03843030 r14: 00007ffd80a08fa0 r15: 000000000000007c
  di: 0000000000000002  si: 00007ffd80a084d0  bp: 00007ffd80a08720  bx: 00007f54b66b71c0
  dx: 0000000000000000  ax: 0000000000000000  cx: 00007f54bef0100b  sp: 00007ffd80a084d0
  ip: 00007f54bef0100b efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
Traceback (most recent call last):

  File /usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3397 in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  Input In [6] in <cell line: 5>
    dfi.export(d, 'df_image.png', fontsize=32, max_rows=-1)

  File ~/.local/lib/python3.9/site-packages/dataframe_image/_pandas_accessor.py:48 in export
    return _export(

  File ~/.local/lib/python3.9/site-packages/dataframe_image/_pandas_accessor.py:117 in _export
    img_str = converter(html)

  File ~/.local/lib/python3.9/site-packages/dataframe_image/_screenshot.py:188 in run
    img = self.take_screenshot()

  File ~/.local/lib/python3.9/site-packages/dataframe_image/_screenshot.py:140 in take_screenshot
    img = mimage.imread(buffer)

  File /usr/local/lib/python3.9/site-packages/matplotlib/image.py:1560 in imread
    with img_open(fname) as image:

  File /usr/local/lib/python3.9/site-packages/PIL/ImageFile.py:112 in __init__
    self._open()

  File /usr/local/lib/python3.9/site-packages/PIL/PngImagePlugin.py:676 in _open
    raise SyntaxError("not a PNG file")

  File <string>
SyntaxError: not a PNG file

I thought the problem was caused by not having a chromedriver path specified in the code, so I tried to add it to the code as follow:

import pandas as pd
import dataframe_image as dfi

d = pd.DataFrame({'one':[1,1,1,1], 'two':[2,2,2,2]})
dfi.export(d, 'df_image.png', fontsize=32, max_rows=-1, chrome_path='/usr/local/bin/chromedriver')

the output of the code would be:

Starting ChromeDriver 90.0.4430.24 (4c6d850f087da467d926e8eddb76550aed655991-refs/branch-heads/4430@{#429}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

adding chromedriver path to the code made the code runs forever without getting any result.

Then I decided to download the latest version of chromedriver and use it, but I kept getting the same output where the code runs forever without getting any results.

Please help on this issue as I've sent multiple emails to the support team for the past 4 days and the replies I'm getting are not solving my issue.

Thanks, Ziyad

ziyad | 7 posts | Dec. 28, 2022, 6:20 p.m. | permalink

It looks like the module is trying to start a thread to run chrome, but threads are not permitted in web apps on PythonAnywhere. Check the documentation for the module to see if it's possible to configure it to not try to create new threads.

glenn | 9498 posts | PythonAnywhere staff | Dec. 29, 2022, 1:08 p.m. | permalink

I fixed the threads issues by changing the number of workers to 1 instead of 5

def get_pdf_data(file_name, p):
    try:
        from asyncio import run
    except ImportError:
        from ._my_asyncio import run

    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        future = executor.submit(run, main(file_name, p))
    return future.result()

The next issues is where chromedriver is working but it doesn't produce anything, it just runs forever

Starting ChromeDriver 90.0.4430.24 (4c6d850f087da467d926e8eddb76550aed655991-refs/branch-heads/4430@{#429}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

I thought the problem might be because only local connections are allowed, so I set the connection to be all IPs , but same issue presisted where the chromedriver is working but it runs forever without producing anything ...

Starting ChromeDriver 90.0.4430.24 (4c6d850f087da467d926e8eddb76550aed655991-refs/branch-heads/4430@{#429}) on port 9515
All remote connections are allowed. Use an allowlist instead!
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

Any idea how to resolve it ?? Could it be a security restriction where port 9515 is not allowed?

Thanks, Ziyad

ziyad | 7 posts | Dec. 30, 2022, 12:29 p.m. | permalink

No, I don't think it's a security restriction -- we don't have anything like that.

What's the exact code that you have right now, and which line is it hanging on? It might also be interesting to know if the same code runs in a console.

giles | 11788 posts | PythonAnywhere staff | Dec. 30, 2022, 4:23 p.m. | permalink

this is the code i'm running, it gets stuck in the last line when exporting the image.

FYI, jupyter notebook is not working for me for some reason, so issues occurred after running the code from the console.

import pandas as pd
import dataframe_image as dfi

d = pd.DataFrame({'one':[1,1,1,1], 'two':[2,2,2,2]})
dfi.export(d, 'df_image.png', fontsize=32, max_rows=-1, chrome_path='/home/ziyad/foreigner_changes/chromedriver' )

output:

Starting ChromeDriver 109.0.5414.25 (771113d280dd3dda2fb422a6c805f0eb2b8ee6ed-refs/branch-heads/5414@{#303}) on port 9515
All remote connections are allowed. Use an allowlist instead!
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

note that it says "All remote connections are allowed. Use an allowlist instead!" so it shouldn't be limited to local connections.

I added lines 117 to 121 to _screenshot.py

115         with open(temp_img, "wb") as f:
116             args = [
117                 "--allowed-ips=''",
118                 "--disable-dev-shm-usage",
119                 "--whitelisted-ips=",                                                                                                                                                            
120                 "--disable-web-security",
121                 "--allow-running-insecure-content",
122                 "--enable-logging",
123                 "--disable-gpu",
124                 "--headless",
125                 "--no-sandbox",
126                 "--crash-dumps-dir=/tmp",
127                 f"--force-device-scale-factor={self.device_scale_factor}",
128             ]

I also added lines 75 to 79 to _browser_pdf.py

def launch_chrome():
 72     chrome_path = get_chrome_path()
 73     args = [
 74         chrome_path,
 75         "--allowed-ips=''",
 76         "--disable-dev-shm-usage",
 77         "--whitelisted-ips=",                                                                                                                                                                    
 78         "--disable-web-security",
 79         "--allow-running-insecure-content",
 80         "--headless",
 81         "--disable-gpu",
 82         "--run-all-compositor-stages-before-draw",
 83         "--remote-debugging-port=9222",
 84         "--crash-dumps-dir=/tmp",
 85     ]
 86     p = Popen(args=args)
 87     return p

and I commented out some lines as mentioned below:

 97 def get_pdf_data(file_name, p):
 98     try:
 99         from asyncio import run
100     except ImportError:                                                                                                                                                                          
101         from ._my_asyncio import run
102 
103   #  with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
104    #     future = executor.submit(run, main(file_name, p))
105     return run

Thanks,
Ziyad

ziyad | 7 posts | Dec. 30, 2022, 4:48 p.m. | permalink

One thing that looks odd to me in that code is that you have

chrome_path='/home/ziyad/foreigner_changes/chromedriver'

I would expect that path to point to the Chrome binary, not to Chromedriver.

giles | 11788 posts | PythonAnywhere staff | Dec. 30, 2022, 7:02 p.m. | permalink

Is Chrome binary already installed in my image where I can point to it in the code?

If it's not installed, can you guide me through the steps to do so?

Thanks,
Ziyad

ziyad | 7 posts | Dec. 30, 2022, 7:23 p.m. | permalink

There's chromium pre-installed -- you can find the path to the executable by running which chromium in the Bash console.

pafk | 2973 posts | PythonAnywhere staff | Dec. 31, 2022, 10:44 a.m. | permalink

When I tried chromium in the code

import pandas as pd
import dataframe_image as dfi

d = pd.DataFrame({'one':[1,1,1,1], 'two':[2,2,2,2]})
dfi.export(d, 'df_image.png', fontsize=32, max_rows=-1, chrome_path='/usr/bin/chromium')

It gives the following error:

[1231/162312.483221:ERROR:bus.cc(393)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[1231/162312.484361:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
[1231/162312.534973:ERROR:gpu_init.cc(426)] Passthrough is not supported, GL is disabled
[1231/162313.665835:INFO:headless_shell.cc(620)] Written to file /tmp/tmpqae4x9xk/temp.png.

Thanks,
Ziyad

ziyad | 7 posts | Dec. 31, 2022, 4:26 p.m. | permalink

Any update?

ziyad | 7 posts | Jan. 2, 2023, 10:25 a.m. | permalink

The lines that you commented out are the lines that run the thread that does the work that you are requesting. So, while that does mean that the code no longer starts threads, it also means that it does not do anything.

glenn | 9498 posts | PythonAnywhere staff | Jan. 2, 2023, 2:52 p.m. | permalink