Is Python appropriate language for the developing high-load systems?
The short answer - yes. The reasons are below:
- There are well-knows systems (like YouTube) that shows Python suitability: YouTube Architecture
- The most performance problems are related with not a language speed, but with communication and databases speed. Let me show the example from my own experience: I've worked on project that process a huge amount of data (about inserting 100-200k records a day, and selecting from about 100-200 million records 10-20 times per second). The bottleneck was a database, not the language speed. All indexes were pretty complex, and selecting can take up to 30-90 seconds which was inappropriate. We had to change architecture: use data pool, caches, AMQP (with RabbitMQ server), and after that we don't know any problems at all with performance.
- Right tools means everything. For example, if you have to serve enormous web-requests, just use the suitable webserver like Tornado. Especially it is useful for serving Comet-based web applications.
- use special servers for dispatching static content (nginx, lighttpd);
- use caching as much as possible (memcache);
- use load balancers;
- use database clusters, replications etc