I’d say a solution will be highly dependent of what exactly you need to do. A solution to write thousands of records per second might be very different from incrementing a counter in the example you provided. More so, there could be no tables at all to handle such load. Consistency/availability requirements are also missing … Read more
We should specify that we talk about horizontal scalability of physical machines — that’s the only problem. CPUs on one machine will be handled by one VM, no matter what the number of those is. node = machine. To begin, I can say that 30-60 nodes you get out of the box (vanilla OTP installation) … Read more
For years, most unix/linux kernels serialize response to accept(2)s, in other words, only one thread is waken up if more than one are blocking on accept(2) against a single open file descriptor. OTOH, many (if not all) kernels still have the thundering herd problem in the select-accept pattern as you describe. I have written a … Read more
google.com, update.microsoft.com, and other services which handle astonishingly high aggregate bandwidth do much of their magic via DNS. BGP Anycast routing is used to announce the IP address of their DNS servers from multiple points around the world. Each DNS server is configured to resolve google.com to IP addresses within a data center which is … Read more
If you’re starting off on a single server, then many advantages of NoSQL go out the window. The biggest advantages to the most popular NoSQL are high availability with less down time. Eventual consistency requirements can lead to performance improvements as well. It really depends on your needs. Document-based – If your data fits well … Read more
Assuming you have both Python’s celery and django-celery installed, create the following tasks.py file under your app: utils/tasks.py from celery import task # other imports @task() def create_user(data): user = User.objects.create_user( username=data[‘username’], email=None, password=data[‘password’] ) user.save() profile = UserProfile() profile.user = user profile.token = generate_token() profile.save() return None Delete your utils/utilities.py file in your example … Read more
It’s a question of degree. The advantages to using a functional language for functional programming are the carrot and the stick. The carrot is that functional languages have functional syntax and semantics and come with functional libraries. The stick is that functional languages can force you to adhere to certain standards. If you do FP … Read more
If you can use something like CRDT, do so. You should get much better performance. It also enables interesting use cases such as working offline and then merging later. However it is not always possible to design things such that a CRDT will work for you. In that case, paxos can solve the hard problems … Read more
Redis is a (sort of) in-memory noSQL database; but I found that my copy (running on linux) dumps to /var/lib/redis/dump.rdb