Learn Node.js by building a backend framework - Velocy

[!IMPORTANT] This repository is part of a larger, more in-depth, and advanced book: NodeBook.

This entire series will be included as a subchapter in the book. I recommend reading NodeBook on the official website or through its GitHub repository.

You can access the current version of the book in the chapters directory or in PDF format (both Light and Dark modes are available) by clicking here. Note that this version includes the current release of the content, and is not the final version.

---

Note

If you're not familiar with javascript, you may also check out my other repository - Learn Javascript - The Easy Way that takes you on a deep and a fun journey into Javascript - from the very basics to the advanced concepts that you'd ever need, without diving into too much theory. Only practical code examples.

---

To master a new concept, it's often best to begin from the ground up. This isn't just another Node.js guide; it's a comprehensive, code-along experience aimed at building a real world product that may be used by thousands of developers. The product that we're going to build will be a backend framework, that too from scratch.

You won't just learn how Node.js works, but also why it operates in a particular way. The guide also includes discussions on relevant data structures and design patterns.

The book also includes a wide range of exercises specifically created to challenge you, that may require commitment and consistent effort on your part. The first exercises start from chapter 7

This guide goes beyond the basics. We're focused on delivering a modular, optimized backend framework that is close to being production-ready. Topics like performance optimization, security measures, and various testing approaches will be covered to ensure the framework is both reliable and extendable.

I highly recommend actively coding alongside this guide, rather than just reading through it, for a full understanding of Node.js and its more intricate aspects.

The repo for our backend framework- Velocy. (W.I.P)

Table of contents

• (Optional) Node.js is way faster than you think
  • Contenders for the test
    • Elysia - Bun
    • Axum - Rust
    • Express - Node.js
    • Velocy - Node.js
  • The benchmark
  • Source code
    • Elysia - Bun
    • Axum - Rust
    • Express - Node.js
    • Velocy - Node.js
  • Results - Typical benchmark
    • Result: Elysia - Bun/Zig (149,047 req/s)
    • Result: Axum - Rust (208,938 req/s)
    • Result: Express - Node.js (28,923 req/s)
  • Result: Velocy - Node.js (83,689)
  • Graphs
    • Latency
    • Requests/sec
    • Idle memory
    • Memory under constant load
    • Verdict - Typical benchmark
  • The real benchmark
    • Updating our code
    • Elysia
    • Express
  • Velocy
  • Results - A real-world use-case
    • Result: Express - Node (50,275 req/sec)
    • Result: Velocy - Node (138,956 req/sec)
    • Latency
    • Latency without max latency bar
    • Requests/sec
    • Idle memory
    • Memory under constant load
  • Final Verdict
• What the heck is a web server any way?
  • Parts of a Web Server:
    • Navigating the World of Protocols: A Quick Overview
    • The Relationship Between HTTP and TCP: Ensuring Reliable Web Communication
    • 1. Data Integrity and Order
    • 2. Acknowledgment Mechanism
    • 3. Complex Interactions
    • 4. Transmission Overhead
  • Asking and Getting: How Web Servers Respond to Your Requests
    • The Request:
    • The Response:
• Your first node.js program
  • What exactly is node or nodejs?
  • Your first node.js program
    • How does console.log() work in Node.js?
    • The process Object:
    • The stdout property of the process object:
• Working with files
  • What will the logging library do
  • How do you work with files anyway?
  • Let’s get back to files
  • A little more about file descriptors
  • Creating our first file
  • path argument
  • flag argument
  • mode argument
  • Reading from a file
  • A small primer on for..of and for await..of in javascript
    • for..of
    • for await..of
  • Reading the json file
  • Buffers
    • Parsing the json file
• logtar our own logging library
  • Initializing a new project
  • A little about SemVer
  • Creating a LogLevel class
  • The Logger class
    • Encapsulation with private fields
  • The LogConfig class
  • Design patterns
    • The Builder pattern
  • Using builder pattern with the LogConfig class
  • jsdoc comments
  • The RollingConfig class
    • The RollingSizeOptions class
    • The RollingTimeOptions class
  • Finishing up the RollingConfig class
    • Let's recap
  • Adding more useful methods in the LogConfig class
    • Why readFileSync?
  • Refactoring the code
    • The Need for Refactoring
    • Creating Separate Files
    • The index.js file
    • The lib/logtar.js file
    • The lib/logger.js file
    • The lib/config/log-config.js file
    • The lib/config/rolling-config.js file
    • The lib/utils/log-level.js file
    • The lib/utils/rolling-options.js class
  • Writing logs
    • 1. Re-using the File Handle
    • 2. Log Rotation
    • 3. Asynchronous Logging
    • 4. Getting Caller Information (Module and Line Number)
    • Testing our current API
    • Implementing logging methods
    • DRY (Don't Repeat Yourself)
    • The log method
    • Considering the log_level member variable
    • Writing to a file
    • Another gotcha
    • Logs directory configuration
    • The require object
    • Adding a new helper to create log directory
    • Updating the init method
    • Completing the log method
  • Capturing metadata
    • What is a Stack?
    • Examples of Stacks
    • The Call Stack
    • Getting the stack info
    • Getting the callee name and the line number
    • A more ergonomic way
    • Using the get_caller_info function
  • A small intro to async vs sync
    • The Balance between Opposites
    • Mixing Asynchronous and Synchronous Code
    • Faster I/O out of the box
    • Blocking Code
    • Concurrency
  • Adding Rolling File Support
    • Rolling features
    • The rolling_check() method
    • file_handle.stat()
    • Calling the rolling_check method
    • A big gotcha!
  • Stack traces across await points
    • Testing the new Log file creation
• HTTP Deep dive
  • A small web server
    • Starting our web server
    • Testing our web server
    • Testing with cURL
  • HTTP Verbs, Versioning and the benefits of HTTP/1.1
    • GET - Retrieve data
    • POST - Create something
    • PUT - Replace or Create
    • HEAD - Retrieve Metadata
    • DELETE - Remove from existence
    • PATCH - Partial updates
    • A small recap
    • The / path
    • HTTP/0.9
    • HTTP/1.0
    • HTTP/1.1
  • User agents
    • User-Agent can be weird
  • MIME Type and Content-Type
    • Understanding the Accept Header
    • Mime Type
    • Anatomy of a MIME type
    • But why the wildcard */*?
    • The Content-Type header
    • The charset=UTF-8: character encoding
  • Headers
    • Header Name
    • Colon (:)
    • Header Value
    • Whitespace
    • Custom X- based headers
  • Request headers
    • Accept
    • Referer
    • Authorization
    • Cookie
    • Host
    • Content-Type (request)
  • Response Headers
    • Content-Type (response)
    • Cache-Control
    • Set-Cookie
  • Response and Status Codes
    • Connection: close in action
    • Status Codes
• Velocy - Our backend framework
  • Why Velocy?
  • What is a backend framework/library anyway?
  • Core features of our backend framework
    • Routing and URL Handling:
    • Middlewares
    • Building our own database
    • Caching
    • Rate limiting
    • Some other features that we will be implementing
  • A basic Router implementation
    • A Toy Router
    • Transfer-Encoding: chunked
    • Chunks, oh no!
    • Specifying Content-Length
    • Code reusability
  • The Router class
    • Using Router with an HTTP server
  • this is not good
    • Lexical Context
    • Arrow functions are not free
    • Why should we care about memory?
    • Testing the updated code
  • Improving the Router API
  • The Need for a Trie
    • What is a Trie anyway?
  • Ex. Implementing a Trie
    • Root Node
    • End of the word
    • Challenge 1: Basic Trie with insert Method
    • Challenge 2: Implement search method
  • Ex. Implementing our Trie based Router
    • Challenge 1: Implementing the addRoute method
    • Challenge 2: Implementing the findRoute method
  • Ex. Adding HTTP method support
    • Requirements
    • More details
    • Example
    • Hints
    • Solution
  • Adding HTTP methods to the Router
    • Update the TrieRouter class
  • Ex. Implementing Dynamic Routing
    • Why Dynamic Routing?
    • Anatomy of a dynamic route
    • Challenge: Enhance the TrieRouter Class to Support Dynamic Routing
    • Visualisation of our TrieRouter structure
    • Summary
  • Running Our Server
    • Refactoring the TrieRouter class
    • Type Aliases
    • The run function
  • Building our first web-server
    • More refactoring
    • Your first web server
  • Ex. Query Parameters (Advanced)
    • Anatomy of a URL with Query Parameters
    • Challenge 1: Implementing Basic Query Parameter Parsing
    • We need to tackle more edge cases
    • Challenge 2: Parsing Query Parameters Manually