2023-07-14

How Deno works

There has been a lot of talk lately about Deno. In this blog post I will share my notes about how Deno works based on me exploring their source code.

If you are looking for an introductory post about Deno then you could read my earlier What is Deno blog post.

"The easiest, most secure JavaScript runtime."

Let's start with their website. Deno is a CLI tool and to use it we need to download Deno executable.

Deno's promise is to provide all the essential features out of the box. The binary provides us with:

  • evaluating TypeScript code
  • formatting the code
  • linting the code
  • type checking the code
  • language server
  • and more.

What's more Deno is completely open source so we can freely explore what it includes and how it works on GitHub.

All of the findings below are for Deno v1.35.0

Deno CLI is a Rust application that uses clap crate for parsing command-line arguments and flags.

> deno --version
deno 1.35.0 (release, aarch64-apple-darwin)
v8 11.6.189.7
typescript 5.1.6

Deno bundles a V8 engine which executes JavaScript. TypeScript compiler is also embedded into the binary to provide type checking (more on this later).

Code formatting

deno fmt source

Deno scans all files in parallel and can either format a file in place or check if a file formatted. Deno can format Markdown, JSON and TypeScript/JavaScript files.

Code linting

deno lint source

Deno uses deno_ast crate to parse the source file which in turn uses swc_ecma_ast crate. Deno traverses the program AST and applies lint rules.

  • All lint rules
  • Default lint rules: rules that have a recommended tag

Example non-default lint rule: "Eqeqeq". It requires triple equals. source

Type checking

deno check main.ts source

Deno builds a module graph first (loads all files). If there is a lock file then verify integrity of modules. If node built-ins are used then inject @types/node typings. Finally, transform the graph to feed to tsc (official TypeScript compiler).

Deno runs a JSRuntime with deno_cli_tsc extension to evaluate tsc and type check the files. Deno includes a complete tsc source code (version 5.1.6). This is what we saw earlier when we printed deno --version.

Code evaluation

deno run main.ts

Now comes the most interesting bit. How does Deno run TypeScript code?.

  1. Find a file to run. Could be stdin, a file or a url.
  2. Build allowed permissions. Permissions you specified when running a command (e.g --allow-net, --allow-read, etc).
  3. Create a MainWorker
    • Set up module loader, NPM resolver and other utilities
    • Check if entry point is a node, NPM or an ES module
      • Module starts with node:.. or npm:..
    • Create a JSRuntime

deno run is not the only way you can evaluate the code. There is also deno repl and deno compile. The latter bundles a Deno script and all of its dependencies into a standalone executable and uses eszip v2 file format for storing the module graph.

JSRuntime

I have already mentioned JSRuntime a couple of times. JSRuntime is an abstraction from deno_core that wraps a V8 isolate and evaluates JavaScript code.

V8 orchestrates isolates: lightweight contexts that evaluate the code in a safe scoped environment. Rust integration is provided by v8 crate.

More about V8 isolates here and here.

JSRuntime can execute a script and evaluate an ES module. JSRuntime lets you to integrate Rust with JS via extensions (more on this later).

Loading a main module will load the module and all of its dependencies. Modules are loaded with a deno_core::ModuleLoader.

Extensions

Deno by default loads a plenty of extensions. Some of these extensions exist in separate crates (meaning you can import and use them), while others are kind of private and exist solely for Deno CLI usage.

Runtime extensions could be defined with a macro:

deno_core::extension!(
  deno_url,
  deps = [deno_webidl],
  ops = [
    op_url_reparse,
    op_url_parse,
    op_url_get_serialization,
    op_url_parse_with_base,
    op_url_parse_search_params,
    op_url_stringify_search_params,
    op_urlpattern_parse,
    op_urlpattern_process_match_input
  ],
  esm = ["00_url.js", "01_urlpattern.js"],
);

This macro defines a deno_url extension that depends on deno_webidl extension. deno_url provides 8 operations (op_url...) that are defined in Rust. JavaScript can call these functions. When runtime initializes the extension it should execute 00_url.js and 01_urlpattern.js scripts.

Now you can call Rust functions from JavaScript:

globalThis.Deno.core.op_url_parse(href, componentsBuf);

Rust side receives the arguments and an optional global state:

#[op(fast)]
pub fn op_url_parse(state: &mut OpState, href: &str, buf: &mut [u32]) -> u32 {
  // implementation that fills the buffer `buf`
}

Operations defined on the Rust side could be synchronous (like op_url_parse) and also asynchronous.

Available extensions:

"Private" extensions from Deno CLI:

JavaScript implementations for these extensions are here.

Snapshots

JSRuntime supports snapshots. Deno CLI uses a pre-built snapshot every time it creates a JSRuntime. Snapshot is built automatically when compiling the crate.

The snapshot's goal is to speed up initial startup time.

Web workers

When launching a web worker Deno starts a new JSRuntime (a new V8 isolate). The process is very similar to executing a main module. Web worker communicates with a main worker via futures_channel.

Build your own runtime

Deno makes it fairly trivial to implement custom runtimes using deno_core crate. The process is essentially:

  1. Create a JSRuntime
    • You want to use a new runtime for every request/user/customer so as to keep them separated.
  2. Register extensions
    • Implement the API you need to be accessible from JavaScript side.
  3. Implement a module loader
    • Loader finds a module and its dependencies.
    • Deno provides a file system loader out of the box.
  4. Now you are good to go.

In the future it might be possible to sandbox code execution using ShadowRealm.

Here are a few helpful links:

Deno Deploy

You may think that Deno Deploy is based on Deno CLI. But it is not the case. Deno Deploy is a proprietary JS runtime implementation built on top of deno_core.