JVM on AWS Cloud: Investigation and Thoughts
Amazon has been the largest cloud service vendor for quite a while.
AWS Lambda is a service that lets you focus on business requirements instead of server details. It together with other AWS components, make it easy to deploy and maintain your business logic.
If you do not care about vendor lock-in, AWS suite may cover most of trivial business requirements. Although you may run into cases where more customization is required, this can be (partly) solved by adding abstraction layer into the architecture of your application.
One more thing to take care of is pricing. Many AWS services charge by usage. It may seem to be very cheap at first, but when your business grows, the pricing may not be delightful.
In addition, you have to be very careful not to leak your credential information. I have witnessed an incident that an employee leaked the code containing AWS access key to GitHub (which is illegal in the first place). Later on the access key was detected by a cracker who abused it to create a hundred of Amazon EC2 instances. Several hours later, the company received report from Amazon and shut down the instances.
Of course the same security rule applies if you are managing physical servers by yourself. However, with a physical machine you may power it off in the worst case, but you can do nothing for AWS instances.
Anyway, AWS is a bless and a curse. Evaluate your requirements and use it wisely.
AWS Lambda Internals: Cold Start and Warm Up
AWS lambda is actually a container running upon a Linux image customized by Amazon1, there is no magic in it.
When a Lambda functions is triggered for the first time, the corresponding container is initialized and started. This process is called cold start. It may take up to several seconds to finish according to the programming language that Lambda function was developed in.
Afterwards, the container will be kept running for some time, so that the cold start will be avoided for the following requests. During this time, the upcoming requests will be handled immediately. If the Lambda function is not triggered for some time, the container will be inactive.
Unfortunately, there is no way to control this process, since it is managed by AWS.
JVM and AWS Lambda: Unavoidable High Latency
Java has much larger memory footprint and longer start time compared with other light-weight languages like Node.js. These two problems become much more obvious if cold start is taken into consideration.
A JVM container might take several seconds to start, according to the memory allocated to the Lambda function. Generally speaking, for a given Java Lambda function, bigger memory leads to faster initialization process2.
(Amazon: Want more speed? Shut up and bring your money!)
In order to solve the cold start problem, there is a technique called warm up. The key point is to create a CloudWatch rule that triggers the target Lambda function periodically to keep the container alive. Please note that this is not really reliable because AWS is a black box.
So if your application is written in Java, you have to bear with the really long cold start time that may cause high latency, which might not be satisfying if your application is user oriented.
Languages with Fast Starting Process
In the end of 2018, AWS announced the C++ Lambda runtime3 that declares single digit millisecond4. But it is relatively hard to write highly reliable C++ code. Also the development speed of C++ is relatively low compared with other popular high level languages.
Node.js has the shortest start time among all supported languages except C++. If your application is not heavily computation oriented, it might be a good idea to consider Node.js. Otherwise, "heavy-weighed" languages like Java or C# has better raw performance.