Docs in the Shell

TL;DR: To access module documentation in the Erlang shell use docsh.

Hi! I’m Radek Szymczyszyn, an Erlang dev. This is my first post here. You can also read it on Erlang Solutions’ blog.

For quite some time now I’ve been envying a few of Elixir’s features: protocols, a consistent standard library, the pipe operator, macros and online documentation (as in “when connected to a system”). Personally, I don’t like Elixir’s syntax, but I “love” it when people use this same argument as an excuse not to use Erlang. Joking apart, some of these features could be present in Erlang, but they are not yet.

Therefore, I want to present you with an idea, one which hopefully will alleviate one of Erlang’s deficiencies — the lack of shell-accessible docs, anywhere and everywhere a piece of code written in the language is deployed. So without further ado, here be Docs in the Shell or docsh for short:

> recon:h().
Recon, as a module, provides access to the high-level functionality contained in the Recon application.
...
> recon:h(get_state, 1).
Shorthand call to recon:get_state(PidTerm, 5000)
ok
> recon:h(get_state, 2).
Fetch the internal state of an OTP process.
Calls sys:get_state/2 directly in R16B01+, and fetches
it dynamically on older versions of OTP.
ok
>

Though some may argue that a REPL or a language-specific shell is just a toy unsuitable for real work, or a learning tool at best, I dare to disagree. Using the Erlang shell we can inspect live production systems, trace system components, profile, initiate hot code reloading, and if a dire need arises, perform surgical cuts to solve performance or scalability emergencies.

Fiddling in the shell is necessary when troubleshooting a system and docs for the system might not be available as manpages (for basically any library apart from OTP), or you might not have them installed, or the project documentation site is down, or you have a crappy connection, or you’re in an enterprise where the only host you can access is Jira and no regular internet access… Or simply none of the above is quick enough, or convenient enough, for checking the information you need: whether it’s ets:lookup(Tab, Key) or ets:lookup(Key, Tab). For me this is enough rationale for easy shell access to docs.

To remind my future self and let you know of the intended project direction, I tried to clarify some goals:

1. Provide added value to existing community projects and libraries as well as OTP. There are a number of tools I use daily and I’d like to equip them with shell doc access as frictionless and efficient as possible. These are community built Recon and Cowboy, but also the “batteries included” into OTP such as dbg, lists, and proplists.
2. Require minimum fuss to enable this in a project. Either an erl_opts directive in rebar.config or inclusion of a single header file should be enough to embed docs, respectively, in all or select modules of a project.
3. Elixir iex doesn’t provide access to documentation for Erlang modules. If possible, make docsh-embedded documentation compatible with Elixir’s format for iex to use.

There are some issues outstanding — the first of them might be perpetual, I’m slightly afraid, the second one is definitely solvable, though I’m not convinced yet as to what the best approach would be:

1. Different projects use disparate documentation formats and even OTP isn’t standardized on the issue — some modules have inline docs, some use external XML. Documentation extraction will require multiple converters from the various formats and maybe some heuristics to extract documentation not formatted according to any particular format at all. In the long run it would be ideal if the community converged to a single format.
2. The storage format in use aims for compatibility with Elixir, but it’s not there yet. The docs aren’t stored as an “ExDc” chunk the way Elixir does it, as it’s not possible to affect the layout of a .beam file at the parse tree transformation phase and that’s when docsh support code generation happens. I’m leaning towards a Rebar3 post-compile plugin to embed the chunk into already-compiled .beam files. As of now, they’re simply embedded as a generated __docs/0 function.

Thanks go to Tomek Kowal, Joe Yiasemides, and Simon Zelazny for initial discussions and encouragement, and to the team at Erlang Solutions in Kraków, where I work and where this idea first sparked in my head.

All feedback is appreciated, so don’t hesitate to reach out to me on Twitter. If you want to help by getting your hands dirty, then this way please, to the GitHub docsh project ;)