Twisted + SQLAlchemy and the best way to do it
In the intervening couple of years, Alex Gaynor created https://github.com/alex/alchimia which may be a better central repository for doing integration with SQLAlchemy and Twisted.
twisted
How to write a Twisted client plugin
Should you perhaps be using the tac files to run it from command line? http://twistedmatrix.com/documents/12.2.0/core/howto/application.html#auto5 Then using your MyChat class should be easier in another program…
How can I find memory leaks in my Python program? [duplicate]
Generally, failing to close cursors is one of the most common kinds of memory leaks. The garbage collector can’t see the MySQL resources involved in the cursor. MySQL doesn’t know that the Python side was released unless the close() method is called explicitly. Rule of thumb. Open, use and close cursors in as short a … Read more
Django Push HTTP Response to users [closed]
HTTP is inherently a “pull” protocol–i.e., a client pulls data from a server, waits around for a while and then pulls more data later. There’s actually no strictly HTTP way to “push” data to a client from a server. You have basically three options when you need to “push” to a client. (1) Do polling–use … Read more
Run a Scrapy spider in a Celery Task
The twisted reactor cannot be restarted. A work around for this is to let the celery task fork a new child process for each crawl you want to execute as proposed in the following post: Running Scrapy spiders in a Celery task This gets around the “reactor cannot be restart-able” issue by utilizing the multiprocessing … Read more
Force python to use an older version of module (than what I have installed now)
A better version of option B. would be to replace import twisted by import pkg_resources pkg_resources.require(“Twisted==8.2.0”) import twisted which will arrange for the correct version of twisted to be imported, so long as it’s installed, and raises an exception otherwise. This is a more portable solution. This won’t work, though (nor would any other variaton … Read more
Twisted server crashes unexpectedly while running django
There is no 499 http code in rfc. Nginx defines 499 code itself. When a client sent a request, and closed the connection without waiting for the response, a 499 code occurs. If there’re a lot of 499s in your access_log, it’s mostly caused by the slow back-ends (too slow for your users to wait). … Read more
Handling large file uploads with Flask
I think the super simple way to get around that simply sends the file in lots of small parts/chunks. So there are going to be two parts to making this work, the front-end (website) and backend (server). For the front-end part, you can use something like Dropzone.js which has no additional dependencies and decent CSS … Read more
Python packages not installing in virtualenv using pip
The problem here is that you’re using sudo when you shouldn’t be. And that’s causing pip to try to install into /usr/local/lib instead of ~/glenv/lib. (And, because you used sudo, it’s successfully doing so, but that doesn’t help you, because you’re not allowing system site-packages in your venv.) There are multiple reasons sudo pip could … Read more
ReactorNotRestartable error in while loop with scrapy
By default, CrawlerProcess‘s .start() will stop the Twisted reactor it creates when all crawlers have finished. You should call process.start(stop_after_crawl=False) if you create process in each iteration. Another option is to handle the Twisted reactor yourself and use CrawlerRunner. The docs have an example on doing that.