AWS recently announced that users can now configure up to 10 GB of ephemeral storage for each Lambda function instance. This new feature will benefit customers running data workloads, like financial analysis, machine learning (ML) inference, or media processing. Ephemeral storage has always been a feature of AWS Lambda. What’s new in AWS Lambda, is the ability to configure ephemeral storage up to 10 GB!
What is Ephemeral Storage?
The temporary, volatile storage attached to instances, present only during the instance’s running lifetime, is known as ephemeral storage. If there’s any issue with the underlying hardware or the instance is terminated or stopped, the data on ephemeral storage will be lost.
Earlier, when using AWS Lambda functions, one could only use ephemeral storage of 512 MB, for each function executed. This was regardless of memory configuration or runtime.
The increased ephemeral storage brings several benefits to the table for users. Let’s have a look at them.
New in AWS Lambda: Increased Ephemeral Storage with Significant Advantages
If you use AWS Lambda to process large files (like images and PDFs) or carry out ETL (extract, transform, and load) jobs on a huge dataset, or operationalize ML models, it’s difficult to follow the runtime storage requirements.
Earlier, to increase the local storage, users needed to use Amazon S3 (Amazon Simple Storage Service) as intermediary storage for reading or writing data at the time of execution. Alternatively, they had to mount an Amazon EFS (Amazon Elastic File System) to the AWS Lambda function.
Although this solved storage issues, users still had to face lowered performance through cold start. They also had to bear substantial extra costs for using EFS. Moreover, there was the general complexity arising from the need to manage several moving pieces.
The newly introduced increased ephemeral storage serves as the solution for these problems. Owing to this feature, users are now able to configure temporary storage between 512 MB and 10 GB, for each execution.
With this new update in AWS Lambda, users can access a low-latency, secure temporary file system in Lambda functions. Additionally, they now have the flexibility to reconfigure the storage, as needed, up to 10 GB.
