MLscript

What would TypeScript look like if was designed with type inference, soundness, and pattern matching in mind?

This original question lead to the creation of the MLscript programming language, a modern object-oriented and functional programming language for the Web.

Since then, the goals of the language have evolved, and MLscript has become a more ambitious effort at defining a next-generation high-level programming language, aiming to maximize reliability, expressiveness, and performance. We are notably adding new backends, such as WASM, so this is no longer "just" a scripting language. Hence, the name of the next iteration of the language will also most likely change.

This repository contains a few iterations of the MLscript experimental compiler, and most notably:

• MLscript Compiler Version 1
• MLscript Compiler Version 2 (hkmc2)

The latter is in active development, on the hkmc2 branch. The continuous integration (CI) has been set up to only test this version of the compiler. The mlscript branch contains the last commit where the CI was set up to test the old version-1 compiler.

Most notably, hkmc2 does not yet reimplement type checking (outside of the experimental InvalML mini-type checker, which constitutes the artifact of our OOPSLA '25 paper). We are working on a new type checker that will be more expressive and more efficient than the old one, but still need to finish other tasks before it can be integrated.

The webpage at https://hkust-taco.github.io/mlscript/ still demonstrates the old version-1 compiler. An online demo of hkmc2 (already somewhat outdated) can be found at https://mlscript.fun/.

MLscript Compiler Version 1

This legacy version of the compiler provided basic JavaScript code-gen along with advanced type checking for object-oriented and functional programming with records, generic classes, mix-in traits, first-class unions and intersections, instance matching, and ML-style principal type inference. These features can be used to implement expressive class hierarchies as well as extensible sums and products.

The approach supports union, intersection, and complement (or negation) connectives, making sure they form a Boolean algebra, and adds enough structure to derive a sound and complete type inference algorithm.

Many extensions were also added over time and splintered off into supporting research paper artifacts, such as more advanced extensible programming support, expressive pattern matching syntax and compilation, and first-class polymorphism (see: Publications).

Getting Started

Project Structure

• The shared/src/main/scala/mlscript directory contains the sources of the MLscript compiler.

• The shared/src/test/scala/mlscript directory contains the sources of the testing infrastructure.

• The shared/src/test/diff directory contains the actual tests.

• The ts2mls/js/src/main/scala/ts2mls directory contains the sources of the ts2mls module.

• The ts2mls/js/src/test/scala/ts2mls directory contains the sources of the ts2mls declaration generation test code.

• The ts2mls/jvm/src/test/scala/ts2mls directory contains the sources of the ts2mls diff test code.

• The ts2mls/js/src/test/typescript directory contains the TypeScript test code.

• The ts2mls/js/src/test/diff directory contains the declarations generated by ts2mls.

• The ts2mls experimental sub-project allows one to use TypeScript libraries in MLscript. It can generate libraries' declaration information in MLscript by parsing TypeScript AST, which can be used in MLscript type checking.

Prerequisites

You need JDK supported by Scala, sbt, Node.js, and TypeScript to compile the project and run the tests.

We recommend you to install JDK and sbt via coursier. The versions of Node.js that passed our tests are from v16.14 to v16.17, v17 and v18. Run npm install in the repository root to install required npm packages (including TypeScript). Note that ScalaJS cannot find globally installed TypeScript. We explicitly support TypeScript v4.7.4.

Running the tests

Running the main MLscript tests only requires the Scala Build Tool installed. In the terminal, run sbt mlscriptJVM/test.

Running the ts2mls MLscript tests requires NodeJS, and TypeScript in addition. In the terminal, run sbt ts2mlsTest/test.

You can also run all tests simultaneously. In the terminal, run sbt test.

Running tests individually

Individual tests can be run with -z. For example, ~mlscriptJVM/testOnly mlscript.DiffTests -- -z parser will watch for file changes and continuously run all parser tests (those that have "parser" in their name).

You can also indicate the test you want in shared/src/test/scala/mlscript/DiffTests.scala:

  // Allow overriding which specific tests to run, sometimes easier for development:
  private val focused = Set[Str](
    // Add the test file path here like this:
    "shared/src/test/diff/mlscript/Methods.mls"
  ).map(os.RelPath(_))

To run the tests in ts2mls sub-project individually, you can indicate the test you want in ts2mls/js/src/test/scala/ts2mls/TSTypeGenerationTests.scala:

private val testsData = List(
    // Put all input files in the `Seq`
    // Then indicate the output file's name
    (Seq("Array.ts"), "Array.d.mls")
  )

Running the web demo locally

To run the demo on your computer, compile the project with sbt fastOptJS, then open the local_testing.html file in your browser.

You can make changes to the type inference code in shared/src/main/scala/mlscript, have it compile to JavaScript on file change with command sbt ~fastOptJS, and immediately see the results in your browser by refreshing the page with F5.

MLscript Compiler Version 2 (hkmc2)

This is the second iteration of the MLscript compiler, nicknamed hkmc2 (Hong Kong MLscript Compiler v2).

Getting Started

Project Structure

Most of the important code of the new compiler is in the hkmc2 folder.

Prerequisites

You need JDK supported by Scala, sbt, Node.js, and TypeScript to compile the project and run the tests.

We recommend you to install JDK and sbt via coursier. The versions of Node.js that passed our tests are from v16.14 to v16.17, v17 and v18.

Run npm install in the repository root to install the required npm packages. This installs TypeScript (used by the JS backend tests) as well as Binaryen (used by the WASM backend tests in hkmc2/shared/src/test/mlscript/wasm/). If you skip this step, WASM tests will fail at runtime because the binaryen module cannot be found by Node.js.

Note that ScalaJS cannot find globally installed TypeScript. We explicitly support TypeScript v4.7.4.

Some tests in the compiler subproject generate and compile C++ while making use of some libraries. You can run these by installing nix (for MacOS, we recommend https://determinate.systems/posts/graphical-nix-installer/) and running nix develop before launching SBT. If you don't want to use nix, you can install the dependencies manually as follows, but this has not been tested on non-MacOS systems:

brew install mimalloc boost gmp

Running the tests

Running the tests requires the Scala Build Tool (SBT) installed.

We recommend running all tests in the SBT shell, i.e., do not restart SBT every time, but launch it in shell mode (with command sbt) and then use one of the following commands.

Main SBT test commands (use these for validating changes before pushing)

• hkmc2JVM/test for running only the main compilation tests, in hkmc2/shared/src/test/mlscript-compile.
• hkmc2DiffTests/test for running only the main diff-tests, in hkmc2/shared/src/test/mlscript.
• hkmc2MainTests/test for running the above two.
• hkmc2AppsTests/test for running the applications compile and diff tests.
• hkmc2NofibTests/test for running the nofib compile and diff tests.
• hkmc2WasmTests/test for running the wasm compile and diff tests.
• hkmc2MostTests/test for running all of the above.
• hkmc2AllTests/test for running all default hkmc2 tests, including ScalaJS-compiled tests.
• ~hkmc2DiffTests/Test/run for running the test watcher, which updates test files as you save them and recompiles the Scala sources automatically on change.
• test for compiling all JVM and JS subprojects and running every single test in the repository.

Individual tests can be run with -z. For example, ~mlscriptJVM/testOnly mlscript.DiffTests -- -z parser will watch for file changes and continuously run all parser tests (those that have "parser" in their name).

Useful SBT commands aliases (use these for more pleasant development)

The most useful command alias for development is watch for continuously running the test watcher, which:

• watches for any file-save event in compilation and diff- tests, updating them accordingly
• recompiles the Scala sources automatically on change

The command aliases below let you either specify no arguments, in which case all corresponding tests are run, or specify a string to filter the tests to run.

• ctest aka "main compilation tests" for running specific tests in hkmc2/shared/src/test/mlscript-compile
• dtest aka "main diff-tests" for running only the main diff-tests in hkmc2/shared/src/test/mlscript
• You can find other similar command aliases in build.sbt.

The command aliases below are shorthand for running aggregated tests; they take no arguments.

• qtest aka "quick tests" for ctest and dtest together.
• mtest aka "main tests" for runnign most tests, excluding JS compilation + diff- tests (usually sufficient)
• atest aka "all tests" for running all tests, including JS compilation + diff- tests

Example usages:

• dtest opt/ for running the optimization diff-tests in mlscript/opt/.
• catest pars to run all application compilation tests with "pars" in their name, such as mlscript-compile/apps/parsing/Lexer.mls.

Another useful SBT incantation is ; qtest; watch. This command runs all quick tests tests once and then starts the test watcher. This is a useful command to use periodically while making changes to the compiler, to check that you haven't broken anything.

Note that when saved, the special file ChangedTests.cmd will trigger the test watcher to run all tests that currently have unstaged changes in git. This is useful when you have a working subset of tests that you want to run periodically.

Publications

The following research publications notably used various iterations of the MLscript compiler for their implementations. Links to the individual paper artifact repositories are provided at the corresponding paper webpages.

• [OOPSLA '22] MLstruct: Principal Type Inference in a Boolean used v1 with the old ML-like syntax.
• [ECOOP '23] super-Charging Object-Oriented Programming Through Precise Typing of Open Recursion (🏆 Distinguished Artifact!) used v1 with the current Scala-like syntax and a new object system.
• [OOPSLA '23] Getting into the Flow: Towards Better Type Error Messages for Constraint-Based Type Inference used v1 with an OCaml-syntax parser and heavily modified type checker.
• [POPL '24] When Subtyping Constraints Liberate: A Novel Approach to Type Inference for First-Class Polymorphism used v1 with the old ML-like syntax.
• [OOPSLA '24] The Ultimate Conditional Syntax (🏆 Distinguished Paper!) used v1 with the current Scala-like syntax; the parser had many problems – the reimplementation of the Ultimate Conditional Syntax in hkmc2 is of much higher quality.
• [GPCE '24] Seamless Scope-Safe Metaprogramming Through Polymorphic Subtype Inference used v1 with the current Scala-like syntax.
• [OOPSLA '25] A Lightweight Type-and-Effect System for Invalidation Safety used hkmc2.