Firecracker Customer Survey results

[xmarcalx]

The scope of this discussion is to present the results of the customer survey distributed to all the members of Firecracker Community available on GiHub and in our general public slack channel.

All the tables can be found in the attached document (see Appendix).

At the moment of writing, Firecracker achieved:

1. about 17.5k stars in GitHub
2. about 1.2k forks of our repo
3. as far as Firecracker team is aware, 15 open-source users who integrate Firecracker in their offerings
4. 2120 members joining our slack channel

An attempt to extract a summary of the status of our open source community can be obtained considering the following data across all the repos within our firecracker-microvm organization since its inception in2018:

1. Number of commits (see Table 1.14)
2. Number of contributors (see Table 1.13) where a contributor is defined as author of a commit. Where possible, the contributors have been sanitized to remove bots (noreply in the email).

From both graphs, clearly stand out that the AWS team is still the main contributor of the overall quantity of commits landed in our repo therefore in general the community consume deliverables more than joining the deliver process itself. Such trend, justify the pattern where even if the overall contributors steadily decreased from Q3 2021 onward, the number of commits published is more or less constant.

We wanted to take some time to learn from the collective experience of everyone to make you more successful in what you do with Firecracker.
This survey was aimed to objectively collect and group the different experiences across our customers by answering the following questions:

1. what do they like or dislike or need from the Firecracker product?
2. what do they like or dislike or improve about how Firecracker team operate?
3. what do they like or dislike or improve about how we engage with them?

Survey Stat

The results listed in this document are based on the collection of 13 responses. Such number, compared the total amount of users, remind us that not all the Firecracker users may be represented by the results presented below.

The Product

In this section, we analyze Firecracker as a product evaluating its features and performances.
The population of Firecracker users who responded to the survey is young, most of our users still use Firecracker from less than 1 year and this is easily understandable because firecracker is a relative young product (see Table 1.1).

From the survey, Firecracker seems to be used in product running in the following different Cloud providers:

1. Amazon Web Services
2. Google Cloud Platform
3. Microsoft Azure
4. VMware
5. Hetzner Cloud

Firecracker currently is used with different CPU family architectures: Intel and AMD.

Features

Firecracker is attractive thanks to the quick VM start time but other compelling reasons for our users involve the capability to take snapshots, the low memory overhead and the support of multi tenancy (see Table 1.2)

All the features are going to be used consistently also in the next 12 months.

At the same time the jailer currently is loosing traction because add complexity in testing and sometimes it gets completely substituted by ad-hoc solutions.

Pain Point

Following, there is a list of features which our open-source user requested:

1. extend compatibility for snapshots across different architectures
   1. Already covered in our Roadmap by [Snaps] Secure snapshot sharing #2940
2. dynamic resize of memory and or CPU in microVMs
3. support for virtiofs (or similar)
   1. There is already an issue with some info on virtiofs [WIP] Implement VirtioFS device backend #1351
4. Support more than 4 vcpus
   1. Firecracker already support up to 32 vcpus

Performance

Firecracker clearly is performing according expectations or more in the following properties (see Table 1.3):

1. microVM start time
2. memory overhead

Increasing microVM density is beneficial for the products in which Firecracker is used (see Table 1.4) but our open source users are not currently limited by Firecracker.

Pain Point

At the same time. Firecracker didn’t progress enough from its initial limitation with I/O reading and writing which still are not satisfying expectations while the microVM networking bandwidth capability has mixed data (see Table 1.3).

The limits on IO performance particularly emerge as a main source of pain because our open source members reported a solid amount of usecases which relate with the capability to perform builds and therefore IO intense workloads.

Developing with Firecracker

In this section, we analyze the experience of integrating and using Firecracker.
Firecracker is not a standalone service or platform, it is a piece of SW which needs to be integrated and tested with the overall architecture of a service.

Integration

From the data, integrating Firecracker is a long process which takes around one year to be completed (see Table 1.5).
Such time is spent mainly to design and implement the control logic (see Table 1.6) but the Firecracker team is supporting its customers quite well (see Table 1.7).

Pain point

The time spent during the integration unfortunately is not frustration free and currently the main pain point arises from (see Table 1.8) :

1. Better support of Firecracker in the Go SDK
2. Add support for CLI users instead of implementing only an HTTP API
3. Improve documentation

Releases

From the data seems that every new single release is used by internal customers (see Table 1.9) which therefore can be considered as a multiplier factor of any pain point in the integration of new releases.

To help the consumption of new release, the users would like to be notified in the slack channel when a new release is available and have additional documentations and improve the release cadence.

Engagement

The Firecracker team manged to maintain a very good communication level with the users (see Table 1.10) but there are isolated signals of dissatisfaction with the communication within the community which the team aim to address with a more consistent approach in our communication channels.

The community is eager to to be engaged in Tech Talks (see Table 1.11) on a quarterly basis (see Table 1.12) and regular office hours were requested with a schedule of 1 time per month to address regularly users' questions and give them the possibility to seek a more close support.

Conclusion

Firecracker it is a product that is mainly attractive for the fast boot time that offer for its microVM and the small overhead.
Firecracker resulted a flexible enough product to be deployed for different use cases and on different cloud providers and consequentially types of servers.
There is a clear set of ask which would improve the experience of our open source users: some already covered by our roadmap while others which needs to be evaluated.
Firecracker Team can improve the engagement with the community.

Appendix

All the tables are presented in the attached file "Firecracker Customer Survey Tables.pdf"