Forums

Hello Pythonanywhere staff / community

My problem is as follows: I am using tensorflow, and I have two pre trained models which I am running, which I can only get to run on the server using different tensorflow versions for each model (one works in 1.9, the other in 2.4). After a full day of trying to get them to run on the same version of tensorflow with some compatibility options, I am convinced it is impossible.

So I then proceeded to try to install two versions of tensorflow in the same python interpreter. My thought is that if I take the tensorflow folder in site-packages and manually move it to tensorflow_1, then install the other version of tensorflow with pip, and then manually move that (in site-packages) to tensorflow_2, and then import them something like this: 'import tensorflow from tensorflow_1' and import tensorflow from 'tensorflow_2'. but, this doen't work becaue the packages reference their own folders as 'tensorflow.something', so after my manual renaming and importing tensorflow, i get a bunch of errors like "tensorflow.something" was not found. My thought is that there may be some way to use relative import paths like 'import thing from otherthing' to get this to work, or else it seems I'd have to rename all of the 'import tensorflow.this' inside the tensorflow .py files to "import tensorflow_1.this". python relative imports are still very confusing to me, and normally I avoid the headache by using always abolute imports. does this sound like a viable solution if I can get some kind of relative import working for the two tensorflow versions? I've checked with pip list after installing both, and they can both can exists with the same other dependencies like identical numpy versions etc.

Having failed to use two versions of tensorflow in one python interpreter, I decided to try using two python interpreters, and have them communicate with eachother is my only remaining option. To my knowlege, the most effective way to achieve this is with sockets, and running to python interpreters on my machine, I can get them to communicate (I think very quickly) by opening a socket. However, from reading some forum posts on here, it seems that using sockets is not posssible on pythonanywhere. So I attempted to have them communicate by reading / writing to some files, and reading / writing to an object in the database. The speed of the entire process is very important for my project. These methods both are extremely slow, and the file communication seems unreliable, the two interpreters seem to not be able to find the file even though it is there, or it takes a long time ( I am using the flush() method, which I though should fix that problem ). The database method results in many 'database locked' errors, and is also extremely slow. I think the biggest problem of both the file and the database method is I am running a while loop and constantly checking for an update of the file or the database, which seems to make it very slow.

Additionally, I had both the tensorflow models working in 2.4 on my laptop, and the reason I can only get them to work on different versions on pythonanywhere is because when using the 2.4 compatibility version to use the 1.9 model, there seems to be some reference to /proc/self/exe. On my computer (ubuntu linux) this is available, and on pythonanywhere i see that I cannot 'ls /proc/self/exe'. this raies an error in the tensorflow code and it doesnt work. But I am able to get that model working with tensorflow 1.9 on pythonanywhere.

What I thought would be a simple process of getting a server running tensorflow that works on my laptop moved to my server on pythonanywhere has turned into a nightmare, and I'm not sure what to try at this point, just looking for any advice or ideas on what to do at this point. To me it seems the best option would be manually moving the tensorflow installs to different folders for each version and importing them with some kind of relative import.

any thoughts? thank you

My main recommendation would be to re-generate both models under the same version before using it on PythonAnywhere- that will solve all your problems without crazy hacks.

I'm not sure if either methods you proposed would work. eg: tensorflow uses some backend-- wouldn't both try to use / access the same backend?

Why do you need to merge these two models? It may be wayyy easier to say run two diff webapps or two diff scripts that can then run completely different versions without talking to each other?

Through a lot of trial and error and reading forums, I was actually able to convert the incompatible model to a tflite, and now I have them both running in the newest tensorflow, so that seems to have solved everything!

Great -- thanks for letting us know!