OptimisticJSONParser

🚀 Ultra-high-performance optimistic JSON parser for Swift

A fault-tolerant JSON parser that gracefully handles malformed JSON while delivering exceptional performance. Inspired by simdjson On-Demand parsing techniques.

✨ Features

• 🛡️ Fault-tolerant: Parses incomplete and malformed JSON
• ⚡ High-performance: 20-30 MB/s sustained throughput
• 🎯 On-demand parsing: SIMD-inspired indexing with lazy evaluation
• 💾 Memory efficient: Minimal allocations, reusable parser instances
• 🔧 Zero dependencies: Pure Swift implementation

📦 Installation

Swift Package Manager

Add to your Package.swift:

dependencies: [
    .package(url: "https://github.com/fumito-ito/OptimisticJSONParser.git", from: "0.0.1")
]

Manual Installation

Copy OptimisticJSONParser.swift to your project.

📖 Usage

Basic Usage

import OptimisticJSONParser

let parser = OptimisticJSONParser()

// Parse complete JSON
let result = parser.parse("""
{
    "name": "John",
    "age": 30,
    "skills": ["Swift", "JSON"]
}
""")

print(result) 
// Output: ["name": "John", "age": 30, "skills": ["Swift", "JSON"]]

Optimistic Parsing Examples

let parser = OptimisticJSONParser()

// Missing closing bracket
let incomplete = parser.parse("""["apple", "banana", "cherry"""")
print(incomplete) // ["apple", "banana", "cherry"]

// Incomplete numbers
let incompleteNumber = parser.parse("""{"price": 12.}""")
print(incompleteNumber) // ["price": 12.0]

// Unclosed strings
let unclosedString = parser.parse("""{"message": "Hello world""")
print(unclosedString) // ["message": "Hello world"]

// Mixed malformed JSON
let mixed = parser.parse("""[1, 2, {"key": "value"""")
print(mixed) // [1, 2, ["key": "value"]]

High-Performance Usage

let parser = OptimisticJSONParser() // Reuse for better performance

// Process large datasets
let largeJSON = loadLargeJSONFile()
let startTime = CFAbsoluteTimeGetCurrent()

if let data = parser.parse(largeJSON) {
    let duration = CFAbsoluteTimeGetCurrent() - startTime
    print("Parsed \(largeJSON.count) bytes in \(duration * 1000) ms")
    print("Speed: \(Double(largeJSON.count) / duration / 1024 / 1024) MB/s")
}

🎯 Supported Optimistic Behaviors

🏗️ Architecture

OptimisticJSONParser uses a two-phase approach inspired by simdjson:

Phase 1: Structural Indexing

• SIMD-inspired scanning identifies JSON structural characters
• Creates an index of positions for [, {, ", numbers, etc.
• Handles string boundaries and escape sequences correctly

Phase 2: On-Demand Parsing

• Lazy evaluation - only materializes requested values
• Iterator-based traversal through structural indices
• Direct conversion to Swift native types

Input JSON → Structural Index → On-Demand Parser → Swift Objects
     ↓              ↓                    ↓              ↓
"[1,2,3]"    [0,1,3,5,6]        Value Iterator    [1, 2, 3]

🔍 Error Handling

OptimisticJSONParser prioritizes data extraction over strict compliance:

let parser = OptimisticJSONParser()

// Returns partial data instead of throwing errors
let result = parser.parse("""{"broken": json}""")
// May return ["broken": nil] or partial object

// For strict validation, use Foundation's JSONSerialization
do {
    let strict = try JSONSerialization.jsonObject(with: data)
} catch {
    // Handle strict parsing errors
    let optimistic = parser.parse(string) // Fallback to optimistic
}

⚡ Performance Tips

1. Reuse parser instances:

   let parser = OptimisticJSONParser() // Create once
   for json in jsonStrings {
       let result = parser.parse(json) // Reuse many times
   }
   

2. Avoid string copying:

   // Efficient - direct string parsing
   let result = parser.parse(jsonString)
   
   // Less efficient - unnecessary data conversion
   let data = jsonString.data(using: .utf8)!
   let string = String(data: data, encoding: .utf8)!
   let result = parser.parse(string)
   

3. Profile with large datasets:

   // Measure actual performance with your data
   let iterations = 1000
   let start = CFAbsoluteTimeGetCurrent()
   
   for _ in 0..<iterations {
       _ = parser.parse(yourJSON)
   }
   
   let avgTime = (CFAbsoluteTimeGetCurrent() - start) / Double(iterations)
   

🧪 Testing

The parser includes comprehensive tests covering:

• ✅ Standard JSON compliance
• ✅ Optimistic parsing scenarios
• ✅ Performance benchmarks
• ✅ Edge cases and malformed inputs
• ✅ Memory efficiency validation

Run tests:

swift test

🆚 Comparison

📚 Inspiration

This parser is inspired by:

• simdjson: High-performance JSON parsing techniques
• On-Demand JSON: A Better Way to Parse Documents?: Research paper on lazy JSON parsing
• Real-world needs: Handling imperfect JSON from logs, APIs, and data pipelines

🤝 Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

📄 License

MIT License - see LICENSE for details.

🙏 Acknowledgments

• The simdjson team for pioneering high-performance JSON parsing
• John Keiser and Daniel Lemire for the On-Demand parsing research
• The Swift community for performance optimization insights

---

Made with ❤️ for the Swift community

"Sometimes you need to be optimistic about your JSON" 🌟