homr

homr is an Optical Music Recognition (OMR) software designed to transform camera pictures or PDFs of sheet music into machine-readable MusicXML format. The resulting MusicXML files can be further processed using tools such as musescore.

For a quick try, visit our online demo at homr.site or

You might also want to check out Andromr, an Android app for optical music recognition using homr.

Prerequisites

• Python 3.11 or 3.12
• Poetry or UV
• Optional
  • NVIDIA GPU with CUDA 12.1
  • AMD GPU with ROCm 7.0

Getting started (uv)

The easiest way to get started is using uvx (uv must be installed). Select an inference backend:

• CPU: uvx --from 'homr[cpu]' homr <image>

• NVIDIA CUDA: uvx --from 'homr[cuda]' homr <image>

• AMD ROCm: uvx --python 3.12 --from 'homr[rocm]' homr <image>

• The resulting MusicXML file will be saved in the same directory as the input image

Getting started (poetry)

• Clone the repository
• Install dependencies for:
  • Inference: poetry install --only main --extras cpu
  • Development: poetry install --extras cpu
  • If using GPU, replace --extras cpu to --extras cuda / --extras rocm
• Run the program using poetry run homr <image>
• The resulting MusicXML file will be saved in the same directory as the input image

Run on multiple images

• Just add your image to the command, for example: poetry run homr <image_1> <image_2>
• This produces one file named merged_image_1.musicxml
• To combine MusicXML files, you can use relieur

Example

The example below provides an overview of the current performance of the implementation. While some errors are present in the output, the overall structure remains accurate.

The homr result is obtained by processing the homr output and rendering it with musescore.

Preprocessing

Depending on the input, preprocessing the image can improve the results. A common step is to remove the background from scanned images.

Limitations

The current implementation focuses on pitch and rhythm information on the bass or treble clef, neglecting dynamics, articulation, double sharps/flats, and other musical symbols.

Technical Details

homr uses a two-stage pipeline: segmentation for structural analysis followed by semantic symbol recognition via transformer models.

Stage 1: Image Segmentation and Structural Analysis

homr employs UNet-based segmentation models (adapted from oemer) to extract structural components from the sheet music image:

• Staff lines and symbols: Detected via trained segmentation networks that identify:
  • Staff line fragments
  • Note heads
  • Stems and rests
  • Bar lines
  • Clefs and key signatures

The segmentation process generates bounding boxes for each detected element. These predictions serve as inputs for the staff detection algorithm.

Stage 2: Staff Detection and Merging

Using the segmentation outputs, homr constructs staffs through the following steps:

1. Staff Anchor Detection: The algorithm identifies "staff anchors" (clefs and bar lines) that serve as reference points for accurate staff localization, even when symbols partially obscure staff lines.

2. Unit Size Estimation: For each staff, the algorithm calculates the "unit size" (distance between staff lines). This accommodates camera perspective variations and non-uniform staff spacing.

3. Staff Reconstruction: Around each anchor, five staff lines are located and the remaining staff structure is reconstructed using the estimated unit size.

4. Grand Staff Merging: Braces and brackets are identified to merge related staffs, supporting:

   • Grand staffs (piano, organ)
   • Multiple voices on a single staff
   • Mixed instrument groups

Stage 3: Semantic Symbol Recognition via Transformer

Each staff is dewarped (perspective-corrected) and passed through a transformer-based model (based on Polyphonic-TrOMR) that performs end-to-end symbol sequence recognition. The model outputs:

• Rhythm symbols: Note durations, rests, and tuplet information
• Pitch information: Absolute pitch values with accidentals (sharps, flats, naturals)
• Articulation marks: Accents, staccato, tenuto, and slur markers
• Performance annotations: Dynamic expressions and other musical notation

The transformer model generates these predictions in sequence, processing the dewarped staff image to understand the spatial and temporal relationships between musical symbols.

Note: The transformer output provides the sequence of symbols but does not include explicit positional information (horizontal or vertical coordinates). However, the model computes the center of attention as a byproduct of the attention mechanism, which is used to estimate the focus point on the staff image. homr maps this point back to the input image and writes it as a comment into each note of the MusicXML output, e.g. <!-- imgpos: 45, 231 -->. These are rough positions: they point at or near the symbol, but aren't pixel-exact.

Stage 4: MusicXML Output

The symbol sequence is converted into MusicXML format and saved to disk. The resulting file can be processed with tools like musescore or relieur (for multi-image combinations).

License

homr is licensed under AGPL-3.0. For commercial use and the licenses of the training data behind the model weights, see LICENSE_FAQ.md.

Citation

If you use this code in your research work, please cite oemer and Polyphonic-TrOMR.

Name

The name "homr" stands for Homer's Optical Music Recognition (OMR), leaving the interpretation of "Homer" to the user's discretion, whether referring to the ancient poet Homer or the iconic character from The Simpsons.

Thanks

This project builds upon previous work, including:

• The segmentation models of oemer
• The transformer model of Polyphonic-TrOMR
• The starter template provided by Benjamin Roland