Is Python appropriate language for the developing high-load systems?
The short answer - yes. The reasons are below:
- There are well-knows systems (like YouTube) that shows Python suitability: YouTube Architecture
- The most performance problems are related with not a language speed, but with communication and databases speed. Let me show the example from my own experience: I've worked on project that process a huge amount of data (about inserting 100-200k records a day, and selecting from about 100-200 million records 10-20 times per second). The bottleneck was a database, not the language speed. All indexes were pretty complex, and selecting can take up to 30-90 seconds which was inappropriate. We had to change architecture: use data pool, caches, AMQP (with RabbitMQ server), and after that we don't know any problems at all with performance.
- Right tools means everything. For example, if you have to serve enormous web-requests, just use the suitable webserver like Tornado. Especially it is useful for serving Comet-based web applications.
- use special servers for dispatching static content (nginx, lighttpd);
- use caching as much as possible (memcache);
- use load balancers;
- use database clusters, replications etc
- High Scalability blog;
- bigdatamatters.com blog;
- Pragmatic Programming Techniques blog.
node.js ftw! :)
ReplyDeleteSorry, you can hate me, but JavaScript is really ugly language. Powerful - yes, flexible - yes, but all its implicit conversions and syntax freedom drive me crazy. Is it normal to allow such-like constructions be valid: "([,Á,È,ª,É,,Ó]=!{}+{},[[Ç,µ]=!!Á+Á][ª+Ó+µ+Ç])()[Á+È+É+µ+Ç](-~Á)" ???
ReplyDeleteCould you please tell where to start from? I'm absolute novice in High Load, so can't even understand what is the beginning of HighLoad and what to search for. Maybe a hint, what should be the first step (I know Python quite good for now) and further direction?
ReplyDelete