[AWS] Having more then 1000 concurrent connections?

[Maciej Sobczak]

On 26/08/2011 00:49, Marius Amado-Alves wrote:

>> I'm not sure. We are *engineers* and we are supposed to be concerned
>> with the physicality of all the bricks that we are using.
>
> Ah the building construction metaphor. How well did that work in the
> past. (Recommended reading here is still The Mythical Manmonth, I
> think.)

Thank you for reading recommendations, but I know systems that do not 
work because their programmers ignored the physicality of what they were 
doing.

> Software engineering is different.

Software engineering does not exist in a vacuum, it relies on hard 
matter (silicon, usually) that sits below it.

> It has abstraction layers.
> That black box thing.

That black box thing is used on paper only - it does not do any actual 
work. However, computers with software do.

> Trying to bring this ship home: the number of concurrent tasks of an
> AWS server. The main factor would be traffic I think. Not the number
> of cores.

The problem is - the traffic capacity is at least in some degree 
*determined* by the number of cores (why do you think companies buy 
bigger servers to handle bigger traffic?), which blows off your 
"abstraction layers" argument.
You cannot just add new tasks with the hope to handle more traffic. At 
some point adding new tasks will have no effect or will even degrade 
performance (hint: scheduler is one story, multiple task stacks 
competing for the cache is another, etc.), so you must stop at some 
value (is it 10? or maybe 100? or maybe 100000?) that is dependent both 
on the structure of your software as well as on your hardware. Finding 
such proper values is an art worth a lot of money, but good starting 
points are based on knowledge that has to be deeper that the "black box 
thing". Separation of concerns in the sense that you have proposed is 
not going to help here.

However, separation of concerns helps in the sense that we don't need to 
know what individual transistors are doing. This certainly makes our 
lives easier.

Regards,

-- 
Maciej Sobczak * www.msobczak.com * www.inspirel.com