English is currently available.
Erlang & Elixir DISCUSSION
My service receives JSON from devices and decodes it to maps with string keys, such as %{"temp" => 21.5, "id" => "A7"}. I would prefer data.temp over data["temp"], so I wrote a helper that converts every key with String.to_atom/1. A reviewer said this can bring down the whole VM.
Is that true, and what is the recommended way to handle keys that come from outside the system?
It is true. Atoms are stored once in a global atom table and referred to by index, which is what makes comparing them so fast. The table is never garbage-collected: an atom, once created, exists until the VM stops. The table also has a fixed capacity, 1,048,576 entries by default, and when it is full the VM does not raise an exception; the whole node terminates.
If outside parties control the key names, they control how many atoms you create. A client that sends 1,000 previously unseen keys per second would fill the default table in about 1,048 seconds, a little over 17 minutes.
The standard approach is to keep external data with string keys and convert at the boundary only the fields you expect. Pattern matching does this neatly: %{"temp" => temp, "id" => id} = payload extracts the values and fails loudly if a key is missing. From there build a struct or a map with atom keys written in your own code, %Reading{id: id, temp: temp}, and use reading.temp inside the application.
That gives a clear rule: atom keys for data whose shape your code defines, string keys for anything whose shape someone else defines.
If you do need a generic conversion, use String.to_existing_atom/1. It succeeds only when the atom already exists, typically because it appears in your compiled code, and raises ArgumentError otherwise, so unknown keys cannot grow the table. The Erlang equivalents are binary_to_existing_atom/2 and list_to_existing_atom/1. JSON libraries usually offer the same choice as a decoding option; check the documentation of the one you use for an "existing atoms only" mode before enabling atom keys.
One caveat: an atom exists only once the module that mentions it has been loaded, so a whitelist kept in the same module that does the conversion is the most predictable arrangement.
You can watch the table to confirm whether you have a leak: :erlang.system_info(:atom_count) returns the number of atoms currently defined and :erlang.system_info(:atom_limit) the capacity. On a healthy system the count rises while code loads and then stays flat. The limit can be raised with the +t emulator flag, but for an unbounded source that only postpones the crash.
Also be careful not to mix key types in one map. %{"temp" => 21.5}[:temp] is nil, because :temp and "temp" are different keys, and that kind of silent miss is a common source of bugs after a half-finished conversion.
Back to Erlang & Elixir community
