๐ An awesome Data Science repository to learn and apply for real world problems. With repository starsโญ and forks๐ด
14
1,241 commits
updated Sep 22, 2026

Contributions are welcome - see CONTRIBUTING.md.
An open-source Data Science repository to learn and apply concepts toward solving real- world problems.
This is a shortcut path to start studying Data Science. Just follow the steps to answer the questions, "What is Data Science, and what should I study to learn Data Science?"
$ brew tap academic/tap
$ brew install academic
Become a sponsor! github@academic.io
Data Science is one of the hottest topics on the Computer and Internet farmland nowadays. People have gathered data from applications and systems until today and now is the time to analyze them. The next steps are producing suggestions from the data and creating predictions about the future. ๐ Here you can find the biggest question for Data Science and hundreds of answers from experts.
| Link | Preview |
|---|---|
ย 37237โญ ย ย 7524๐ด Data Science For Beginners) | Microsoft are pleased to offer a 10-week, 20-lesson curriculum all about Data Science. |
| ๐ What is Data Science @ O'reilly | Data scientists combine entrepreneurship with patience, the willingness to build data products incrementally, the ability to explore, and the ability to iterate over a solution. They are inherently interdisciplinary. They can tackle all aspects of a problem, from initial data collection and data conditioning to drawing conclusions. They can think outside the box to come up with new ways to view the problem, or to work with very broadly defined problems: โhereโs a lot of data, what can you make from it?โ |
| ๐ What is Data Science @ Quora | Data Science is a combination of a number of aspects of Data such as Technology, Algorithm development, and data interference to study the data, analyse it, and find innovative solutions to difficult problems. Basically Data Science is all about Analysing data and driving for business growth by finding creative ways. |
| ๐ The sexiest job of 21st century | Data scientists today are akin to Wall Street โquantsโ of the 1980s and 1990s. In those days people with backgrounds in physics and math streamed to investment banks and hedge funds, where they could devise entirely new algorithms and data strategies. Then a variety of universities developed masterโs programs in financial engineering, which churned out a second generation of talent that was more accessible to mainstream firms. The pattern was repeated later in the 1990s with search engineers, whose rarefied skills soon came to be taught in computer science programs. |
| ๐ Wikipedia | Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from many structural and unstructured data. Data science is related to data mining, machine learning and big data. |
| ๐ How to Become a Data Scientist | Data scientists are big data wranglers, gathering and analyzing large sets of structured and unstructured data. A data scientistโs role combines computer science, statistics, and mathematics. They analyze, process, and model data then interpret the results to create actionable plans for companies and other organizations. |
| ๐ a very short history of #datascience | The story of how data scientists became sexy is mostly the story of the coupling of the mature discipline of statistics with a very young one--computer science. The term โData Scienceโ has emerged only recently to specifically designate a new profession that is expected to make sense of the vast stores of big data. But making sense of data has a long history and has been discussed by scientists, statisticians, librarians, computer scientists and others for years. The following timeline traces the evolution of the term โData Scienceโ and its use, attempts to define it, and related terms. |
| ๐ Software Development Resources for Data Scientists | Data scientists concentrate on making sense of data through exploratory analysis, statistics, and models. Software developers apply a separate set of knowledge with different tools. Although their focus may seem unrelated, data science teams can benefit from adopting software development best practices. Version control, automated testing, and other dev skills help create reproducible, production-ready code and tools. |
| ๐ Data Scientist Roadmap | Data science is an excellent career choice in todayโs data-driven world where approx 328.77 million terabytes of data are generated daily. And this number is only increasing day by day, which in turn increases the demand for skilled data scientists who can utilize this data to drive business growth. |
| ๐ Navigating Your Path to Becoming a Data Scientist | _Data science is one of the most in-demand careers today. With businesses increasingly relying on data to make decisions, the need for skilled data scientists has grown rapidly. Whether itโs tech companies, healthcare organizations, or even government institutions, data scientists play a crucial role in turning raw data into valuable insights. But how do you become a data scientist, especially if youโre just starting out? _ |
While not strictly necessary, having a programming language is a crucial skill to be effective as a data scientist. Currently, the most popular language is Python, closely followed by R. Python is a general-purpose scripting language that sees applications in a wide variety of fields. R is a domain-specific language for statistics, which contains a lot of common statistics tools out of the box.
๐ Python is by far the most popular language in science, due in no small part to the ease at which it can be used and the vibrant ecosystem of user-generated packages. To install packages, there are two main methods: Pip (invoked as pip install), the package manager that comes bundled with Python, and ๐ Anaconda (invoked as conda install), a powerful package manager that can install packages for Python, R, and can download executables like Git.
Unlike R, Python was not built from the ground up with data science in mind, but there are plenty of third party libraries to make up for this. A much more exhaustive list of packages can be found later in this document, but these four packages are a good set of choices to start your data science journey with: ๐ Scikit-Learn is a general-purpose data science package which implements the most popular algorithms - it also includes rich documentation, tutorials, and examples of the models it implements. Even if you prefer to write your own implementations, Scikit-Learn is a valuable reference to the nuts-and-bolts behind many of the common algorithms you'll find. With ๐ Pandas, one can collect and analyze their data into a convenient table format. ๐ Numpy provides very fast tooling for mathematical operations, with a focus on vectors and matrices. ๐ Seaborn, itself based on the ๐ Matplotlib package, is a quick way to generate beautiful visualizations of your data, with many good defaults available out of the box, as well as a gallery showing how to produce many common visualizations of your data.
When embarking on your journey to becoming a data scientist, the choice of language isn't particularly important, and both Python and R have their pros and cons. Pick a language you like, and check out one of the Free courses we've listed below!
If you're just starting out, here's a simple recommended path:
This section contains agent frameworks and tools that are useful for data science workflows.
ย ย ย 679โญ ย ย ย 105๐ด ADK-Rust) - Production-ready AI agent development kit for Rust with model-agnostic design (Gemini, OpenAI, Anthropic), multiple agent types (LLM, Graph, Workflow), MCP support, and built-in telemetry.ย ย ย 312โญ ย ย ย ย 44๐ด Lumen) - Agent framework for chatting with data, turning natural language into SQL, transformation pipelines and visualizations. Outputs are declarative specs that can be inspected, edited, reopened in a notebook or composed into a dashboard.ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด Frostbyte MCP) - MCP server providing 13 data tools for AI agents: real-time crypto prices, IP geolocation, DNS lookups, web scraping to markdown, code execution, and screenshots. One API key for 40+ services.ย ย ย ย ย 1โญ ย ย ย ย ย 1๐ด GitHub)ย ย ย ย ย 8โญ ย ย ย ย ย 1๐ด GitHub)ย ย ย ย 46โญ ย ย ย ย 16๐ด DeepAlpha) - AI crypto trading framework using LightGBM + XGBoost ensemble with 72 ML features. 70.9% walk-forward validated accuracy on out-of-sample data. Supports Bybit and Binance. MIT licensed, available on ๐ PyPI.ย ย ย ย 21โญ ย ย ย ย ย 4๐ด CAJAL) - Local AI agent for generating publication-ready scientific papers with real arXiv citations, IMRaD structure, and tribunal scoring. Runs 100% offline via Ollama with 4B-9B models. MIT licensed. ๐ HuggingFaceย ย ย 120โญ ย ย ย ย 44๐ด ai-evaluation) - Open-source LLM and agent evaluation framework with 50+ metrics, LLM-as-Judge augmentation, and guardrail scanners (jailbreak, PII, prompt-injection). Useful for scoring RAG outputs, agent trajectories, and function-calling behavior in data-science workflows.ย ย ย 292โญ ย ย ย ย 22๐ด Kitaru) - Open-source platform that records real AI agent runs, replays them against changes, and evaluates outcomes before deployment.๐ BGPT MCP - MCP server that gives AI agents access to a database of scientific papers built from raw experimental data extracted from full-text studies. Returns 25+ structured fields per paper including methods, results, sample sizes, and quality scores. ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด GitHub)
ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด Chunk Tuner) - Open-source Python library and MCP server to benchmark document chunking strategies for RAG, score retrieval quality, and recommend configurations for a corpus.
ย ย ย ย 16โญ ย ย ย ย ย 0๐ด II-Commons) - Daily-updated skill and CLI for deterministic retrieval across arXiv, PubMed/PMC, and supported US policy corpora.
๐ Spraay x402 Gateway - x402 payment gateway with 23 Research & Reference endpoints for AI agents: Wikipedia, arXiv, PubMed, Wikidata, academic citation lookup, entity extraction, and more. Pay-per-call in USDC on Base & Solana โ no API keys or subscriptions. Also serves 150+ endpoints across 39 categories including geospatial, AI inference, DeFi, and compute. GitHub
๐ Suppr - AI literature search, document translation, and deep-research workspace for researchers.
How do you learn data science? By doing data science, of course! Okay, okay - that might not be particularly helpful when you're first starting out. In this section, we've listed some learning resources, in rough order from least to greatest commitment - Tutorials, Massively Open Online Courses (MOOCs), Intensive Programs, and Colleges.
ย ย 8406โญ ย ย 2583๐ด #tidytuesday) - A weekly data project aimed at the R ecosystem.ย ย ย 618โญ ย ย ย 253๐ด Data science your way)ย ย ย 701โญ ย ย ย 211๐ด PySpark Cheatsheet)ย ย 1260โญ ย ย ย 454๐ด Tutorials of source code from the book Genetic Algorithms with Python by Clinton Sheppard)ย ย ย ย 85โญ ย ย ย ย 27๐ด Tutorials to get started on signal processing for machine learning)ย 12822โญ ย ย 2040๐ด Minimum Viable Study Plan for Machine Learning Interviews)ย 10751โญ ย ย 1508๐ด Train LLM From Scratch) - A straightforward method for training your LLM, from downloading data to generating text.ย 21975โญ ย ย 4256๐ด Data Science) - Open Source Society Universityย 31248โญ ย ย 2585๐ด AI Expert Roadmap) - Roadmap to becoming an Artificial Intelligence Expertย ย ย 559โญ ย ย ย ย 63๐ด MLSys-NYU-2022) - Slides, scripts and materials for the Machine Learning in Finance course at NYU Tandon, 2022.ย ย ย 889โญ ย ย ย 133๐ด Hands-on Train and Deploy ML) - A hands-on course to train and deploy a serverless API that predicts crypto prices.ย ย 4156โญ ย 30923๐ด Data Science Specialization)ย ย ย 160โญ ย ย ย 195๐ด A list of colleges and universities offering degrees in data science.)This section is a collection of packages, tools, algorithms, and other useful items in the data science world.
These are some Machine Learning and Data Mining algorithms and models help you to understand your data and derive meaning from it.
ย ย ย 658โญ ย ย ย 167๐ด datacompy) - DataComPy is a package to compare two Pandas DataFrames.ย ย ย 955โญ ย ย ย 177๐ด scikit-multilearn)ย ย ย 491โญ ย ย ย ย 71๐ด sklearn-expertsys)ย ย 1586โญ ย ย ย 438๐ด scikit-feature)ย ย ย 421โญ ย ย ย ย 72๐ด scikit-rebate)ย ย ย 706โญ ย ย ย 102๐ด seqlearn)ย ย ย 520โญ ย ย ย 117๐ด sklearn-bayes)ย ย ย 440โญ ย ย ย 206๐ด sklearn-crfsuite)ย ย ย 771โญ ย ย ย 128๐ด sklearn-deap)ย ย ย ย 75โญ ย ย ย ย 11๐ด sigopt_sklearn)ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด sklearn-evaluation)ย ย 6594โญ ย ย 2414๐ด scikit-image)ย ย 6715โญ ย ย 1107๐ด scikit-opt)ย ย ย 388โญ ย ย ย ย 46๐ด scikit-posthocs)ย ย ย ย ย 1โญ ย ย ย ย ย 0๐ด me_fasttext) - Memory-efficient FastText variant with exact trie n-gram IDs, structure-aware row sharing, and mmap serving for large-vocabulary NLP.ย ย ย 667โญ ย ย ย 174๐ด pystruct)ย ย 3089โญ ย ย ย 516๐ด xLearn)ย ย 5286โญ ย ย ย 682๐ด cuML)ย ย 6005โญ ย ย ย 877๐ด causalml)ย ย 5710โญ ย ย 1720๐ด mlpack)ย ย 5175โญ ย ย ย 916๐ด MLxtend)ย ย 2365โญ ย ย ย 322๐ด modAL)ย ย 1150โญ ย ย ย 254๐ด Sparkit-learn)ย ย 2517โญ ย ย ย 168๐ด hyperlearn)ย 14445โญ ย ย 3438๐ด dlib)ย ย 1619โญ ย ย ย 142๐ด imodels)ย ย ย ย 23โญ ย ย ย ย ย 1๐ด jSciPy) - A Java port of SciPy's signal processing module, offering filters, transformations, and other scientific computing utilities.ย ย ย 443โญ ย ย ย 122๐ด RuleFit)ย ย 1015โญ ย ย ย 291๐ด pyGAM)ย ย 4056โญ ย ย ย 302๐ด Deepchecks)ย 28784โญ ย ย 8902๐ด XGBoost)ย 18794โญ ย ย 4070๐ด LightGBM)ย ย 9111โญ ย ย 1335๐ด CatBoost)ย ย ย 708โญ ย ย ย ย 42๐ด PerpetualBooster)ย 36321โญ ย ย 3794๐ด JAX)ย ย ย ย 11โญ ย ย ย ย 22๐ด PhilanthroPy) - Scikit-learn native toolkit for nonprofit fundraising analytics: leakage-safe donor propensity, lapse, planned-giving, wealth-screening and revenue-forecasting estimators.103162โญ ย 30013๐ด PyTorch)ย ย ย 215โญ ย ย ย ย 15๐ด TorchDR) - GPU and multi-GPU dimensionality reduction with a scikit-learn-compatible API.ย 17925โญ ย ย 7265๐ด torchvision)ย ย 3554โญ ย ย ย 807๐ด torchtext)ย ย 2943โญ ย ย ย 800๐ด torchaudio)ย ย 4786โญ ย ย ย 726๐ด ignite)ย ย 1723โญ ย ย ย 310๐ด PyTorchNet)ย ย ย 578โญ ย ย ย ย 63๐ด PyToune)ย ย 6179โญ ย ย ย 417๐ด skorch)ย ย ย 362โญ ย ย ย ย 51๐ด PyVarInf)ย 24098โญ ย ย 4058๐ด pytorch_geometric)ย ย 3915โญ ย ย ย 602๐ด GPyTorch)ย ย 9057โญ ย ย 1021๐ด pyro)ย ย 3385โญ ย ย ย 397๐ด Catalyst)ย ย 1687โญ ย ย ย 179๐ด pytorch_tabular)ย 10609โญ ย ย 3423๐ด Yolov3)ย 58064โญ ย 17466๐ด Yolov5)ย 61883โญ ย 11790๐ด Yolov8)200233โญ ย 77037๐ด TensorFlow)ย ย 7380โญ ย ย 1582๐ด TensorLayer)ย ย 9575โญ ย ย 2352๐ด TFLearn)ย ย 9970โญ ย ย 1305๐ด Sonnet)ย ย 6284โญ ย ย 1778๐ด tensorpack)ย ย 3130โญ ย ย ย 386๐ด TRFL)ย ย 3733โญ ย ย ย 330๐ด Polyaxon)ย ย ย 735โญ ย ย ย 158๐ด NeuPy)ย ย ย 353โญ ย ย ย ย 35๐ด tfdeploy)ย ย ย 706โญ ย ย ย 101๐ด tensorflow-upstream)ย ย 1816โญ ย ย ย 262๐ด TensorFlow Fold)ย ย ย ย 60โญ ย ย ย ย 27๐ด tensorlm)ย ย ย ย 11โญ ย ย ย ย ย 5๐ด TensorLight)ย ย 1629โญ ย ย ย 255๐ด Mesh TensorFlow)ย 11762โญ ย ย 1217๐ด Ludwig)ย ย 3027โญ ย ย ย 753๐ด TF-Agents)ย ย 3305โญ ย ย ย 523๐ด TensorForce)ย ย 1585โญ ย ย ย 635๐ด keras-contrib)ย ย 2173โญ ย ย ย 314๐ด Hyperas)ย ย 1574โญ ย ย ย 303๐ด Elephas)ย ย ย 486โญ ย ย ย ย 47๐ด Hera)ย ย 2397โญ ย ย ย 344๐ด Spektral)ย ย ย 584โญ ย ย ย 108๐ด qkeras)ย ย 5548โญ ย ย 1343๐ด keras-rl)ย ย 1636โญ ย ย ย 264๐ด Talos)ย 33506โญ ย ย 3184๐ด Netron)ย ย ย ย ย 7โญ ย ย ย ย ย 0๐ด Resseract Lite)ย ย 2036โญ ย ย ย ย 87๐ด vizzu)ย ย 3471โญ ย ย ย 360๐ด TensorWatch)ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด github.com)ย ย ย 765โญ ย ย ย ย 32๐ด torchvista) - Interactive notebook-based tool to visualize the forward pass of any PyTorch model.ย ย ย ย 25โญ ย ย ย ย ย 2๐ด FlexViz) - Python library for interactive, cross-filtered dashboards that stay responsive on 100M+ rows by aggregating with Polars on the server.| Link | Description |
|---|---|
ย ย ย 529โญ ย ย ย ย 75๐ด The Data Science Lifecycle Process) | The Data Science Lifecycle Process is a process for taking data science teams from Idea to Value repeatedly and sustainably. The process is documented in this repo |
ย ย ย 205โญ ย ย ย ย 60๐ด Data Science Lifecycle Template Repo) | Template repository for data science lifecycle project |
ย ย ย 570โญ ย ย ย ย 82๐ด TabGAN) | Synthetic tabular data generation using GANs, Diffusion Models, and LLMs with adversarial filtering and privacy metrics. |
ย ย ย 277โญ ย ย ย ย 24๐ด RexMex) | A general purpose recommender metrics library for fair evaluation. |
ย ย ย 785โญ ย ย ย 100๐ด ChemicalX) | A PyTorch based deep learning library for drug pair scoring. |
ย ย ย ย 30โญ ย ย ย ย ย 5๐ด FileShot.io) | Secure zero-knowledge encrypted file sharing (AES-256-GCM in-browser). No account required, MIT licensed, self-hostable, optional link expiry. |
| ๐ CorpusExplorer | Software for corpus linguists and text/data mining enthusiasts. Build your own corpora in over 60 languages. Use over 50 tools/visualizations. |
ย ย 2990โญ ย ย ย 402๐ด PyTorch Geometric Temporal) | Representation learning on dynamic graphs. |
ย ย ย 715โญ ย ย ย ย 54๐ด Little Ball of Fur) | A graph sampling library for NetworkX with a Scikit-Learn like API. |
ย ย 2286โญ ย ย ย 254๐ด Karate Club) | An unsupervised machine learning extension library for NetworkX with a Scikit-Learn like API. |
ย ย 3546โญ ย ย ย 456๐ด ML Workspace) | All-in-one web-based IDE for machine learning and data science. The workspace is deployed as a Docker container and is preloaded with a variety of popular data science libraries (e.g., Tensorflow, PyTorch) and dev tools (e.g., Jupyter, VS Code) |
ย ย 9649โญ ย ย ย 742๐ด xonsh shell) | A Python-powered shell that enables integration, management and orchestration of data science libraries mostly written in Python, allowing you to build pipelines, code and command-based workflows. It can also be used as a kernel for Jupyter Notebook. |
| ๐ Neptune.ai | Community-friendly platform supporting data scientists in creating and sharing machine learning models. Neptune facilitates teamwork, infrastructure management, models comparison and reproducibility. |
ย ย ย 136โญ ย ย ย ย 32๐ด steppy) | Lightweight, Python library for fast and reproducible machine learning experimentation. Introduces very simple interface that enables clean machine learning pipeline design. |
ย ย ย ย 23โญ ย ย ย ย ย 9๐ด steppy-toolkit) | Curated collection of the neural networks, transformers and models that make your machine learning work faster and more effective. |
| ๐ Datalab from Google | easily explore, visualize, analyze, and transform data using familiar languages, such as Python and SQL, interactively. |
| ๐ Hortonworks Sandbox | is a personal, portable Hadoop environment that comes with a dozen interactive Hadoop tutorials. |
| ๐ R | is a free software environment for statistical computing and graphics. |
| ๐ Tidyverse | is an opinionated collection of R packages designed for data science. All packages share an underlying design philosophy, grammar, and data structures. |
| ๐ RStudio | IDE โ powerful user interface for R. Itโs free and open source, and works on Windows, Mac, and Linux. |
| ๐ Python - Pandas - Anaconda | Completely free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing |
ย ย 3256โญ ย ย ย 239๐ด Pandas GUI) | Pandas GUI |
ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด NuriStat) | Free open-source SPSS alternative โ menu-driven desktop statistics (t-tests, ANOVA, regression, survival analysis, ROC) with SPSS .sav import/export |
ย 39834โญ ย ย 3121๐ด Polars) | Fast DataFrame library for Rust and Python, designed as a faster alternative to Pandas |
| ๐ CiteMe | free academic citation generator with a built-in reference checker that flags fabricated or hallucinated references. Searches 11+ scholarly databases (OpenAlex, PubMed, Semantic Scholar, CrossRef, SciELO), formats 40+ citation styles, and offers a public API. No sign-up; available in English, Spanish, Portuguese, French, and German. |
| ๐ Scikit-Learn | Machine Learning in Python |
| ๐ NumPy | NumPy is fundamental for scientific computing with Python. It supports large, multi-dimensional arrays and matrices and includes an assortment of high-level mathematical functions to operate on these arrays. |
| ๐ Vaex | Vaex is a Python library that allows you to visualize large datasets and calculate statistics at high speeds. |
| ๐ SciPy | SciPy works with NumPy arrays and provides efficient routines for numerical integration and optimization. |
| ๐ Data Science Toolbox | Coursera Course |
| ๐ Data Science Toolbox | Blog |
| ๐ Wolfram Data Science Platform | Take numerical, textual, image, GIS or other data and give it the Wolfram treatment, carrying out a full spectrum of data science analysis and visualization and automatically generate rich interactive reportsโall powered by the revolutionary knowledge-based Wolfram Language. |
| ๐ Datadog | Solutions, code, and devops for high-scale data science. |
| ๐ Variance | Build powerful data visualizations for the web without writing JavaScript |
| ๐ Kite Development Kit | The Kite Software Development Kit (Apache License, Version 2.0), or Kite for short, is a set of libraries, tools, examples, and documentation focused on making it easier to build systems on top of the Hadoop ecosystem. |
| ๐ Domino Data Labs | Run, scale, share, and deploy your models โ without any infrastructure or setup. |
| ๐ Apache Flink | A platform for efficient, distributed, general-purpose data processing. |
| ๐ Apache Hama | Apache Hama is an Apache Top-Level open source project, allowing you to do advanced analytics beyond MapReduce. |
| ๐ Weka | Weka is a collection of machine learning algorithms for data mining tasks. |
| ๐ Octave | GNU Octave is a high-level interpreted language, primarily intended for numerical computations.(Free Matlab) |
| ๐ Apache Spark | Lightning-fast cluster computing |
ย ย ย 326โญ ย ย ย ย 69๐ด Hydrosphere Mist) | a service for exposing Apache Spark analytics jobs and machine learning models as realtime, batch or reactive web services. |
| ๐ Data Mechanics | A data science and engineering platform making Apache Spark more developer-friendly and cost-effective. |
| ๐ Caffe | Deep Learning Framework |
| ๐ Torch | A SCIENTIFIC COMPUTING FRAMEWORK FOR LUAJIT |
ย ย 3860โญ ย ย ย 804๐ด Nervana's python based Deep Learning Framework) | Intelยฎ Nervanaโข reference deep learning framework committed to best performance on all hardware. |
ย ย ย 396โญ ย ย ย ย 52๐ด Skale) | High performance distributed data processing in NodeJS |
| ๐ Aerosolve | A machine learning package built for humans. |
ย ย ย 312โญ ย ย ย ย 80๐ด Intel framework) | Intelยฎ Deep Learning Framework |
| ๐ Datawrapper | An open source data visualization platform helping everyone to create simple, correct and embeddable charts. Also at ย ย 1458โญ ย ย ย 279๐ด github.com) |
| ๐ Tensor Flow | TensorFlow is an Open Source Software Library for Machine Intelligence |
| ๐ Natural Language Toolkit | An introductory yet powerful toolkit for natural language processing and classification |
ย 20466โญ ย ย 2043๐ด FunASR) | Industrial-grade speech recognition toolkit supporting 50+ languages with built-in VAD, punctuation, speaker diarization, and emotion detection. OpenAI-compatible API server included. |
| ๐ Annotation Lab | Free End-to-End No-Code platform for text annotation and DL model training/tuning. Out-of-the-box support for Named Entity Recognition, Classification, Relation extraction and Assertion Status Spark NLP models. Unlimited support for users, teams, projects, documents. |
| ๐ nlp-toolkit for node.js | This module covers some basic nlp principles and implementations. The main focus is performance. When we deal with sample or training data in nlp, we quickly run out of memory. Therefore every implementation in this module is written as stream to only hold that data in memory that is currently processed at any step. |
| ๐ Julia | high-level, high-performance dynamic programming language for technical computing |
ย ย 2905โญ ย ย ย 426๐ด IJulia) | a Julia-language backend combined with the Jupyter interactive environment |
| ๐ Apache Zeppelin | Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more |
ย ย 7683โญ ย ย ย 915๐ด Featuretools) | An open source framework for automated feature engineering written in python |
ย ย 1536โญ ย ย ย 229๐ด Optimus) | Cleansing, pre-processing, feature engineering, exploratory data analysis and easy ML with PySpark backend. |
ย 15309โญ ย ย 1706๐ด Albumentations) | ะ fast and framework agnostic image augmentation library that implements a diverse set of augmentation techniques. Supports classification, segmentation, and detection out of the box. Was used to win a number of Deep Learning competitions at Kaggle, Topcoder and those that were a part of the CVPR workshops. |
ย 15881โญ ย ย 1329๐ด DVC) | An open-source data science version control system. It helps track, organize and make data science projects reproducible. In its very basic scenario it helps version control and share large data and model files. |
ย ย ย ย 26โญ ย ย ย ย ย 5๐ด Lambdo) | is a workflow engine that significantly simplifies data analysis by combining in one analysis pipeline (i) feature engineering and machine learning (ii) model training and prediction (iii) table population and column evaluation. |
ย ย 7301โญ ย ย 1442๐ด Feast) | A feature store for the management, discovery, and access of machine learning features. Feast provides a consistent view of feature data for both model training and model serving. |
ย ย 3733โญ ย ย ย 330๐ด Polyaxon) | A platform for reproducible and scalable machine learning and deep learning. |
| ๐ UBIAI | Easy-to-use text annotation tool for teams with most comprehensive auto-annotation features. Supports NER, relations and document classification as well as OCR annotation for invoice labeling |
ย ย 6883โญ ย ย ย 801๐ด Trains) | Auto-Magical Experiment Manager, Version Control & DevOps for AI |
ย ย 1307โญ ย ย ย 159๐ด Hopsworks) | Open-source data-intensive machine learning platform with a feature store. Ingest and manage features for both online (MySQL Cluster) and offline (Apache Hive) access, train and serve models at scale. |
ย 39763โญ ย ย 6244๐ด MindsDB) | MindsDB is an Explainable AutoML framework for developers. With MindsDB you can build, train and use state of the art ML models in as simple as one line of code. |
ย ย ย 510โญ ย ย ย 101๐ด Lightwood) | A Pytorch based framework that breaks down machine learning problems into smaller blocks that can be glued together seamlessly with an objective to build predictive models with one line of code. |
ย ย 4119โญ ย ย ย 749๐ด AWS Data Wrangler) | An open-source Python package that extends the power of Pandas library to AWS connecting DataFrames and AWS data related services (Amazon Redshift, AWS Glue, Amazon Athena, Amazon EMR, etc). |
| ๐ Amazon Rekognition | AWS Rekognition is a service that lets developers working with Amazon Web Services add image analysis to their applications. Catalog assets, automate workflows, and extract meaning from your media and applications. |
| ๐ Amazon Textract | Automatically extract printed text, handwriting, and data from any document. |
| ๐ Amazon Lookout for Vision | Spot product defects using computer vision to automate quality inspection. Identify missing product components, vehicle and structure damage, and irregularities for comprehensive quality control. |
| ๐ Amazon CodeGuru | Automate code reviews and optimize application performance with ML-powered recommendations. |
ย ย 4187โญ ย ย ย 345๐ด CML) | An open source toolkit for using continuous integration in data science projects. Automatically train and test models in production-like environments with GitHub Actions & GitLab CI, and autogenerate visual reports on pull/merge requests. |
| ๐ Dask | An open source Python library to painlessly transition your analytics code to distributed computing systems (Big Data) |
ย 41617โญ ย ย 3814๐ด DuckDB) | An in-process SQL OLAP database management system |
| ๐ Statsmodels | A Python-based inferential statistics, hypothesis testing and regression framework |
| ๐ Gensim | An open-source library for topic modeling of natural language text |
| ๐ spaCy | A performant natural language processing toolkit |
ย ย 8809โญ ย ย 1477๐ด Grid Studio) | Grid studio is a web-based spreadsheet application with full integration of the Python programming language. |
ย 49968โญ ย 19121๐ด Python Data Science Handbook) | Python Data Science Handbook: full text in Jupyter Notebooks |
ย ย ย 228โญ ย ย ย ย 34๐ด Shapley) | A data-driven framework to quantify the value of classifiers in a machine learning ensemble. |
| ๐ DAGsHub | A platform built on open source tools for data, model and pipeline management. |
| ๐ Deepnote | A new kind of data science notebook. Jupyter-compatible, with real-time collaboration and running in the cloud. |
| ๐ Valohai | An MLOps platform that handles machine orchestration, automatic reproducibility and deployment. |
| ๐ PyMC3 | A Python Library for Probabalistic Programming (Bayesian Inference and Machine Learning) |
| ๐ PyStan | Python interface to Stan (Bayesian inference and modeling) |
| ๐ hmmlearn | Unsupervised learning and inference of Hidden Markov Models |
ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด Chaos Genius) | ML powered analytics engine for outlier/anomaly detection and root cause analysis |
| ๐ Nimblebox | A full-stack MLOps platform designed to help data scientists and machine learning practitioners around the world discover, create, and launch multi-cloud apps from their web browser. |
ย ย 3450โญ ย ย ย 257๐ด Towhee) | A Python library that helps you encode your unstructured data into embeddings. |
ย ย ย 672โญ ย ย ย ย 58๐ด LineaPy) | Ever been frustrated with cleaning up long, messy Jupyter notebooks? With LineaPy, an open source Python library, it takes as little as two lines of code to transform messy development code into production pipelines. |
ย ย 2233โญ ย ย ย 168๐ด envd) | ๐๏ธ machine learning development environment for data science and AI/ML engineering teams |
| ๐ Explore Data Science Libraries | A search engine ๐ tool to discover & find a curated list of popular & new libraries, top authors, trending project kits, discussions, tutorials & learning resources |
ย ย ย 718โญ ย ย ย ย 42๐ด MLEM) | ๐ถ Version and deploy your ML models following GitOps principles |
| ๐ MLflow | MLOps framework for managing ML models across their full lifecycle |
ย 11669โญ ย ย ย 919๐ด cleanlab) | Python library for data-centric AI and automatically detecting various issues in ML datasets |
ย 10728โญ ย ย 1195๐ด AutoGluon) | AutoML to easily produce accurate predictions for image, text, tabular, time-series, and multi-modal data |
| ๐ Arize AI | Arize AI community tier observability tool for monitoring machine learning models in production and root-causing issues such as data quality and performance drift. |
| ๐ Aureo.io | Aureo.io is a low-code platform that focuses on building artificial intelligence. It provides users with the capability to create pipelines, automations and integrate them with artificial intelligence models โ all with their basic data. |
| ๐ ERD Lab | Free cloud based entity relationship diagram (ERD) tool made for developers. |
| ๐ Arize-Phoenix | MLOps in a notebook - uncover insights, surface problems, monitor, and fine tune your models. |
ย ย ย 176โญ ย ย ย ย 70๐ด Comet) | An MLOps platform with experiment tracking, model production management, a model registry, and full data lineage to support your ML workflow from training straight through to production. |
ย 22188โญ ย ย 1817๐ด Opik) | Evaluate, test, and ship LLM applications across your dev and production lifecycles. |
| ๐ Synthical | AI-powered collaborative environment for research. Find relevant papers, create collections to manage bibliography, and summarize content โ all in one place |
ย ย ย ย 13โญ ย ย ย ย ย 0๐ด teeplot) | Workflow tool to automatically organize data visualization output |
ย 45814โญ ย ย 4387๐ด Streamlit) | App framework for Machine Learning and Data Science projects |
ย 43596โญ ย ย 3608๐ด Gradio) | Create customizable UI components around machine learning models |
ย 11258โญ ย ย ย 900๐ด Weights & Biases) | Experiment tracking, dataset versioning, and model management |
ย 15881โญ ย ย 1329๐ด DVC) | Open-source version control system for machine learning projects |
ย 14830โญ ย ย 1391๐ด Optuna) | Automatic hyperparameter optimization software framework |
ย 43889โญ ย ย 8068๐ด Ray Tune) | Scalable hyperparameter tuning library |
ย 46932โญ ย 17892๐ด Apache Airflow) | Platform to programmatically author, schedule, and monitor workflows |
ย 23893โญ ย ย 2534๐ด Prefect) | Workflow management system for modern data stacks |
ย 11006โญ ย ย 1079๐ด Kedro) | Open-source Python framework for creating reproducible, maintainable data science code |
ย ย 2595โญ ย ย ย 216๐ด Hamilton) | Lightweight library to author and manage reliable data transformations |
ย 25772โญ ย ย 3750๐ด SHAP) | Game theoretic approach to explain the output of any machine learning model |
ย ย 6946โญ ย ย ย 787๐ด InterpretML) | InterpretML implements the Explainable Boosting Machine (EBM), a modern, fully interpretable machine learning model based on Generalized Additive Models (GAMs). This open-source package also provides visualization tools for EBMs, other glass-box models, and black-box explanations |
ย 12163โญ ย ย 1846๐ด LIME) | Explaining the predictions of any machine learning classifier |
ย ย 7547โญ ย ย ย 890๐ด flyte) | Workflow automation platform for machine learning |
ย 13899โญ ย ย 2574๐ด dbt) | Data build tool |
ย ย 2342โญ ย ย ย ย 75๐ด zasper) | Supercharged IDE for Data Science |
ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด skrub) | A Python library to ease preprocessing and feature engineering for tabular machine learning |
ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด Glyph) | Framework-agnostic TypeScript library for generating, searching, and comparing MinHash fingerprints for fast text similarity, deduplication, and retrieval. |
| ๐ Codeflash | Ship Blazing-Fast Python Code โ Every Time |
| ๐ Hugging Face | Popular open platform for sharing ML models, datasets, and collaborating on NLP and generative AI projects. |
ย ย ย ย 73โญ ย ย ย ย ย 6๐ด Chinese-Elite) | An open-source project that automatically maps relationship networks by parsing public data using LLMs and visualizes it as an interactive graph. |
ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด Desbordante) | An open-source data profiler specifically focused on discovery and validation of complex patterns, such as ๐ numerical association rules, ๐ differential dependencies, ๐ denial constraints, and more. |
ย ย ย ย 57โญ ย ย ย ย ย 8๐ด dna-claude-analysis) | Personal genome analysis toolkit with Python scripts analyzing raw DNA data across 17 categories (health risks, ancestry, pharmacogenomics, nutrition, psychology, and more) and generating a terminal-style single-page HTML visualization. |
ย ย ย 257โญ ย ย ย ย 10๐ด RunMat) | Fast MATLAB-syntax runtime with automatic CPU/GPU execution and fused array kernels. |
ย ย ย ย 17โญ ย ย ย ย ย 2๐ด Turbostream) | A terminal UI for experimenting with custom rule engines and selective LLM analysis on real-time data streams, without worrying about streaming infra or backpressure. |
ย ย 1792โญ ย ย ย 166๐ด WFGY ProblemMap) | Open source โfailure atlasโ of 16 recurring issues in LLM and RAG pipelines, with observable symptoms and suggested fixes for data science teams. |
| ๐ Deploybase | Track real-time GPU and LLM pricing across all cloud and inference providers. |
ย ย 4647โญ ย ย ย 736๐ด DeepAnalyze) | An agentic LLM for autonomous data science, which can autonomously complete a wide range of data science tasks without human intervention. |
ย ย ย ย ย 7โญ ย ย ย ย ย 0๐ด Disco) | Superhuman exploratory data analysis. Finds the feature interactions and subgroup effects in tabular data that LLMs and manual exploration miss โ with p-values, effect sizes, and literature citations. Free for public data. |
| ๐ AI for Database | Chat with your database in natural language โ no SQL needed. Get instant insights, build self-refreshing dashboards, and trigger automated workflows based on database changes. |
ย ย ย ย 46โญ ย ย ย ย 16๐ด Crypto Pump Scanner) | AI-powered cryptocurrency trading bot with LSTM neural network (84.6% accuracy). Real-time pump detection, walk-forward validated models, multi-exchange support (Bybit, Binance, OKX, Gate.io). Open source. |
ย ย 2050โญ ย ย ย 632๐ด Future AGI) | Open-source platform to simulate, evaluate, trace, guardrail, route, and optimize LLM and AI agent apps in one feedback loop, so agents don't just get monitored, they self-improve. Self-hostable. Apache-2.0. |
This section includes some additional reading material, channels to watch, and talks to listen to.
๐ Managing ML Projects
๐ Data for All
ย ย 6761โญ 241810๐ด How to Share Data with a Statistician)Below are some Social Media links. Connect with other data scientists!
| Description | |
|---|---|
| ๐ Big Data Combine | Rapid-fire, live tryouts for data scientists seeking to monetize their models as trading strategies |
| Big Data Mania | Data Viz Wiz, Data Journalist, Growth Hacker, Author of Data Science for Dummies (2015) |
| ๐ Big Data Science | Big Data, Data Science, Predictive Modeling, Business Analytics, Hadoop, Decision and Operations Research. |
| Charlie Greenbacker | Director of Data Science at @ExploreAltamira |
| ๐ Chris Said | Data scientist at Twitter |
| ๐ Clare Corthell | Dev, Design, Data Science @mattermark #hackerei |
| ๐ DADI Charles-Abner | #datascientist @Ekimetrics. , #machinelearning #dataviz #DynamicCharts #Hadoop #R #Python #NLP #Bitcoin #dataenthousiast |
| ๐ Data Science Central | Data Science Central is the industry's single resource for Big Data practitioners. |
| ๐ Data Science London | Data Science. Big Data. Data Hacks. Data Junkies. Data Startups. Open Data |
| ๐ Data Science Renee | Documenting my path from SQL Data Analyst pursuing an Engineering Master's Degree to Data Scientist |
| ๐ Data Science Report | Mission is to help guide & advance careers in Data Science & Analytics |
| ๐ Data Science Tips | Tips and Tricks for Data Scientists around the world! #datascience #bigdata |
| ๐ Data Vizzard | DataViz, Security, Military |
| ๐ DataScienceX | |
| deeplearning4j | |
| ๐ DJ Patil | White House Data Chief, VP @ RelateIQ. |
| ๐ Domino Data Lab | |
| ๐ Drew Conway | Data nerd, hacker, student of conflict. |
| Emilio Ferrara | #Networks, #MachineLearning and #DataScience. I work on #Social Media. Postdoc at @IndianaUniv |
| ๐ Erin Bartolo | Running with #BigData--enjoying a love/hate relationship with its hype. @iSchoolSU #DataScience Program Mgr. |
| ๐ Greg Reda | Working @ GrubHub about data and pandas |
| ๐ Gregory Piatetsky | KDnuggets President, Analytics/Big Data/Data Mining/Data Science expert, KDD & SIGKDD co-founder, was Chief Scientist at 2 startups, part-time philosopher. |
| ๐ Hadley Wickham | Chief Scientist at RStudio, and an Adjunct Professor of Statistics at the University of Auckland, Stanford University, and Rice University. |
| ๐ Hakan Kardas | Data Scientist |
| ๐ Hilary Mason | Data Scientist in Residence at @accel. |
| ๐ Jeff Hammerbacher | ReTweeting about data science |
| ๐ John Myles White | Scientist at Facebook and Julia developer. Author of Machine Learning for Hackers and Bandit Algorithms for Website Optimization. Tweets reflect my views only. |
| ๐ Juan Miguel Lavista | Principal Data Scientist @ Microsoft Data Science Team |
| ๐ Julia Evans | Hacker - Pandas - Data Analyze |
| ๐ Kenneth Cukier | The Economist's Data Editor and co-author of Big Data (https://www.big-data-book.com/). |
| Kevin Davenport | Organizer of https://www.meetup.com/San-Diego-Data-Science-R-Users-Group/ |
| ๐ Kevin Markham | Data science instructor, and founder of ๐ Data School |
| ๐ Kim Rees | Interactive data visualization and tools. Data flaneur. |
| ๐ Kirk Borne | DataScientist, PhD Astrophysicist, Top #BigData Influencer. |
| Linda Regber | Data storyteller, visualizations. |
| ๐ Luis Rei | PhD Student. Programming, Mobile, Web. Artificial Intelligence, Intelligent Robotics Machine Learning, Data Mining, Natural Language Processing, Data Science. |
| Mark Stevenson | Data Analytics Recruitment Specialist at Salt (@SaltJobs) Analytics - Insight - Big Data - Data science |
| ๐ Matt Harrison | Opinions of full-stack Python guy, author, instructor, currently playing Data Scientist. Occasional fathering, husbanding, organic gardening. |
| ๐ Matthew Russell | Mining the Social Web. |
| ๐ Mert Nuhoฤlu | Data Scientist at BizQualify, Developer |
| ๐ Monica Rogati | Data @ Jawbone. Turned data into stories & products at LinkedIn. Text mining, applied machine learning, recommender systems. Ex-gamer, ex-machine coder; namer. |
| ๐ Noah Iliinsky | Visualization & interaction designer. Practical cyclist. Author of vis books: https://www.oreilly.com/pub/au/4419 |
| ๐ Paul Miller | Cloud Computing/ Big Data/ Open Data Analyst & Consultant. Writer, Speaker & Moderator. Gigaom Research Analyst. |
| ๐ Peter Skomoroch | Creating intelligent systems to automate tasks & improve decisions. Entrepreneur, ex-Principal Data Scientist @LinkedIn. Machine Learning, ProductRei, Networks |
| ๐ Prash Chan | Solution Architect @ IBM, Master Data Management, Data Quality & Data Governance Blogger. Data Science, Hadoop, Big Data & Cloud. |
| ๐ Quora Data Science | Quora's data science topic |
| ๐ R-Bloggers | Tweet blog posts from the R blogosphere, data science conferences, and (!) open jobs for data scientists. |
| ๐ Rand Hindi | |
| ๐ Randy Olson | Computer scientist researching artificial intelligence. Data tinkerer. Community leader for @DataIsBeautiful. #OpenScience advocate. |
| ๐ Recep Erol | Data Science geek @ UALR |
| ๐ Ryan Orban | Data scientist, genetic origamist, hardware aficionado |
| ๐ Sean J. Taylor | Social Scientist. Hacker. Facebook Data Science Team. Keywords: Experiments, Causal Inference, Statistics, Machine Learning, Economics. |
| ๐ Silvia K. Spiva | #DataScience at Cisco |
| ๐ Harsh B. Gupta | Data Scientist at BBVA Compass |
| ๐ Spencer Nelson | Data nerd |
| ๐ Talha Oz | Enjoys ABM, SNA, DM, ML, NLP, HI, Python, Java. Top percentile Kaggler/data scientist |
| ๐ Tasos Skarlatidis | Complex Event Processing, Big Data, Artificial Intelligence and Machine Learning. Passionate about programming and open-source. |
| ๐ Terry Timko | InfoGov; Bigdata; Data as a Service; Data Science; Open, Social & Business Data Convergence |
| ๐ Tony Baer | IT analyst with Ovum covering Big Data & data management with some systems engineering thrown in. |
| ๐ Tony Ojeda | Data Scientist , Author , Entrepreneur. Co-founder @DataCommunityDC. Founder @DistrictDataLab. #DataScience #BigData #DataDC |
| ๐ Vamshi Ambati | Data Science @ PayPal. #NLP, #machinelearning; PhD, Carnegie Mellon alumni (Blog: https://allthingsds.wordpress.com ) |
| ๐ Wes McKinney | Pandas (Python Data Analysis library). |
| ๐ WileyEd | Senior Manager - @Seagate Big Data Analytics @McKinsey Alum #BigData + #Analytics Evangelist #Hadoop, #Cloud, #Digital, & #R Enthusiast |
| ๐ WNYC Data News Team | The data news crew at @WNYC. Practicing data-driven journalism, making it visual, and showing our work. |
| ๐ Alexey Grigorev | Data science author |
| ๐ ฤฐlker Arslan | Data science author. Shares mostly about Julia programming |
| ๐ INEVITABLE | AI & Data Science Start-up Company based in England, UK |
| ๐ Jan Oliver Rรผdiger | ML, DL and Data Science - with a focus on text-/data-mining |
Some data mining competition platforms
| Preview | Description |
|---|---|
๐ ![]() | ๐ Key differences of a data scientist vs. data engineer |
๐ ![]() | A visual guide to Becoming a Data Scientist in 8 Steps by ๐ DataCamp ๐ (img) |
๐ ![]() | Mindmap on required skills ๐ img) |
๐ ![]() | Swami Chandrasekaran made a ๐ Curriculum via Metro map. |
๐ ![]() | by ๐ @kzawadz via ๐ twitter |
๐ ![]() | By ๐ Data Science Central |
๐ ![]() | Data Science Wars: R vs Python |
๐ ![]() | How to select statistical or machine learning techniques |
| ๐ | ๐ Choosing the Right Estimator |
๐ ![]() | The Data Science Industry: Who Does What |
๐ ![]() | Data Science |
๐ ![]() | Different Data Science Skills and Roles from ๐ Springboard |
๐ ![]() | A simple and friendly way of teaching your non-data scientist/non-statistician colleagues ๐ how to avoid mistakes with data. From Geckoboard's ๐ Data Literacy Lessons. |
ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด College ROI Dataset) - Lifetime return-on-investment estimates for ~30K US bachelor's programs across 1,775 institutions, built from FREOPP, IPEDS, and BEA regional price data. 5 CSVs with data dictionary, CC BY 4.0, Zenodo DOI.ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด AI Displacement Tracker) - Structured dataset tracking 92 AI-attributed workforce reduction events affecting 453,748 workers across 12 countries and 11 sectors. JSON and CSV formats. CC-BY-4.0 licensed.ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด Pokemon Card Centering Measurements) - 320 measured PSA-style centering annotations (left/right and top/bottom border percentages, tilt) across 302 real eBay-listed Pokemon cards. CSV, CC BY 4.0, Zenodo DOI.ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด Pokemon Card Sold-Price Reference by Grade) - Median sold price by grade (raw, PSA 9, PSA 10) for 486 Pokemon cards, with sample size and confidence flag per card. CSV, CC BY 4.0, Zenodo DOI.ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด GitHub), and HuggingFace mirror. Permanent DOI, CC BY 4.0.ย ย ย 523โญ ย ย ย 192๐ด Open Data Sources)ย ย ย 349โญ ย ย ย ย 82๐ด Public Git Archive)ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด NAYN.CO Turkish News with categories)ย ย 1166โญ ย ย ย 596๐ด Covid-19)ย ย ย 117โญ ย ย ย ย 68๐ด Covid-19 Google)ย ย ย 119โญ ย ย ย ย 43๐ด 5000 Images of Clothes)ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด HDD Price Index) - Daily open dataset of the cheapest new internal 3.5" SATA hard-drive price per terabyte (USD/TB) by capacity tier on Amazon US, with a historical time series. CSV, JSON and JSONL, no login, CC BY 4.0.ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด BDE Score) - AI-powered multi-market stock analysis with transparent BDE scoring across 73 stocks (US/HK/A-share). EU AI Act Art.50 compliant. MIT license.ย ย ย 183โญ ย ย ย ย 33๐ด FirstData) - The world's most comprehensive authoritative data source knowledge base. 210+ curated sources from governments, international organizations, and research institutions. MCP integration for AI agents. MIT licensed.ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด latamdata-py) - Python package for one-line access to 38 open research datasets from Latin America (health, neuroscience, mental health, economics). pip install latamdata-py.ย ย ย ย ย 4โญ ย ย ย ย ย 0๐ด ZipCheckup) - Free ZIP-level environmental safety data for 42,000+ US ZIP codes: water quality, air quality, PFAS contamination, radon, lead, flood risk, and 11 more verticals. Public REST API, npm/PyPI packages, CC BY 4.0.ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด Verified Supplement Evidence) - Evidence-graded dietary-supplement dataset covering dosing, bioavailability by form, drug-nutrient interactions, NHANES deficiency prevalence, FDA FAERS adverse-event signals, and cost-per-effective-dose, with every clinical claim citing a PubMed PMID. CC BY 4.0, DOI 10.57967/hf/9356.ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด WhatFontIs-Bench) - Synthetic benchmark for font family identification with 11,995 images of words set in 600 known fonts, annotated with word and per-letter boxes.ย 33679โญ ย ย 3602๐ด awesome-awesomeness)ย 74391โญ ย 15651๐ด Awesome Machine Learning)ย 11504โญ ย ย ย 785๐ด lists)ย ย 4414โญ ย ย ย 471๐ด awesome-dataviz)322202โญ ย 28776๐ด awesome-python)ย 29354โญ ย ย 8016๐ด Data Science IPython Notebooks.)ย ย 6514โญ ย ย 1513๐ด awesome-r)ย 79094โญ ย 11848๐ด awesome-datasets)ย 18216โญ ย ย 3966๐ด awesome-Machine Learning & Deep Learning Tutorials)ย ย ย 693โญ ย ย ย ย 89๐ด Awesome Data Science Ideas)ย 28866โญ ย ย 6144๐ด Machine Learning for Software Engineers)ย ย 6639โญ ย ย ย 830๐ด Awesome Machine Learning On Source Code)ย ย 2451โญ ย ย ย 356๐ด Awesome Community Detection)ย ย 4801โญ ย ย ย 723๐ด Awesome Graph Classification)ย ย 2475โญ ย ย ย 341๐ด Awesome Decision Tree Papers)ย ย 1834โญ ย ย ย 329๐ด Awesome Fraud Detection Papers)ย ย 1051โญ ย ย ย 166๐ด Awesome Gradient Boosting Papers)ย ย ย 543โญ ย ย ย ย 94๐ด Awesome Computer Vision Models)ย ย ย 715โญ ย ย ย ย 76๐ด Awesome Monte Carlo Tree Search)ย ย 3851โญ ย ย ย 559๐ด 100 NLP Papers)ย ย 1124โญ ย ย ย ย 87๐ด Awesome Game Datasets)ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด ML/AI Interview Prep) - 500+ ML/AI interview Q&A with runnable code โ covers ML fundamentals, deep learning, NLP, PyTorch, scikit-learn pipelines, and system designย 10154โญ ย ย 2154๐ด Data Science Interviews Questions)ย ย 1990โญ ย ย ย 136๐ด Awesome Explainable Graph Reasoning)ย ย ย 101โญ ย ย ย ย 15๐ด Awesome Drug Synergy, Interaction and Polypharmacy Prediction)ย ย 2788โญ ย ย ย 645๐ด Data Science Projects)ย ย 1975โญ ย ย ย 284๐ด Awesome Data Analysis) - A curated list of data analysis tools, libraries and resources.ย ย ย ย 27โญ ย ย ย ย ย 9๐ด Awesome Evidence Synthesis) - A curated list of open-source tools for systematic reviews, meta-analysis, and evidence synthesis.ย ย 1502โญ ย ย ย 160๐ด Awesome Music Production)ย 30046โญ ย ย 6641๐ด academic/awesome-datascience)
๐ An awesome Data Science repository to learn and apply for real world problems. With repository starsโญ and forks๐ด
14
1,241 commits
updated Sep 22, 2026

Contributions are welcome - see CONTRIBUTING.md.
An open-source Data Science repository to learn and apply concepts toward solving real- world problems.
This is a shortcut path to start studying Data Science. Just follow the steps to answer the questions, "What is Data Science, and what should I study to learn Data Science?"
$ brew tap academic/tap
$ brew install academic
Become a sponsor! github@academic.io
Data Science is one of the hottest topics on the Computer and Internet farmland nowadays. People have gathered data from applications and systems until today and now is the time to analyze them. The next steps are producing suggestions from the data and creating predictions about the future. ๐ Here you can find the biggest question for Data Science and hundreds of answers from experts.
| Link | Preview |
|---|---|
ย 37237โญ ย ย 7524๐ด Data Science For Beginners) | Microsoft are pleased to offer a 10-week, 20-lesson curriculum all about Data Science. |
| ๐ What is Data Science @ O'reilly | Data scientists combine entrepreneurship with patience, the willingness to build data products incrementally, the ability to explore, and the ability to iterate over a solution. They are inherently interdisciplinary. They can tackle all aspects of a problem, from initial data collection and data conditioning to drawing conclusions. They can think outside the box to come up with new ways to view the problem, or to work with very broadly defined problems: โhereโs a lot of data, what can you make from it?โ |
| ๐ What is Data Science @ Quora | Data Science is a combination of a number of aspects of Data such as Technology, Algorithm development, and data interference to study the data, analyse it, and find innovative solutions to difficult problems. Basically Data Science is all about Analysing data and driving for business growth by finding creative ways. |
| ๐ The sexiest job of 21st century | Data scientists today are akin to Wall Street โquantsโ of the 1980s and 1990s. In those days people with backgrounds in physics and math streamed to investment banks and hedge funds, where they could devise entirely new algorithms and data strategies. Then a variety of universities developed masterโs programs in financial engineering, which churned out a second generation of talent that was more accessible to mainstream firms. The pattern was repeated later in the 1990s with search engineers, whose rarefied skills soon came to be taught in computer science programs. |
| ๐ Wikipedia | Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from many structural and unstructured data. Data science is related to data mining, machine learning and big data. |
| ๐ How to Become a Data Scientist | Data scientists are big data wranglers, gathering and analyzing large sets of structured and unstructured data. A data scientistโs role combines computer science, statistics, and mathematics. They analyze, process, and model data then interpret the results to create actionable plans for companies and other organizations. |
| ๐ a very short history of #datascience | The story of how data scientists became sexy is mostly the story of the coupling of the mature discipline of statistics with a very young one--computer science. The term โData Scienceโ has emerged only recently to specifically designate a new profession that is expected to make sense of the vast stores of big data. But making sense of data has a long history and has been discussed by scientists, statisticians, librarians, computer scientists and others for years. The following timeline traces the evolution of the term โData Scienceโ and its use, attempts to define it, and related terms. |
| ๐ Software Development Resources for Data Scientists | Data scientists concentrate on making sense of data through exploratory analysis, statistics, and models. Software developers apply a separate set of knowledge with different tools. Although their focus may seem unrelated, data science teams can benefit from adopting software development best practices. Version control, automated testing, and other dev skills help create reproducible, production-ready code and tools. |
| ๐ Data Scientist Roadmap | Data science is an excellent career choice in todayโs data-driven world where approx 328.77 million terabytes of data are generated daily. And this number is only increasing day by day, which in turn increases the demand for skilled data scientists who can utilize this data to drive business growth. |
| ๐ Navigating Your Path to Becoming a Data Scientist | _Data science is one of the most in-demand careers today. With businesses increasingly relying on data to make decisions, the need for skilled data scientists has grown rapidly. Whether itโs tech companies, healthcare organizations, or even government institutions, data scientists play a crucial role in turning raw data into valuable insights. But how do you become a data scientist, especially if youโre just starting out? _ |
While not strictly necessary, having a programming language is a crucial skill to be effective as a data scientist. Currently, the most popular language is Python, closely followed by R. Python is a general-purpose scripting language that sees applications in a wide variety of fields. R is a domain-specific language for statistics, which contains a lot of common statistics tools out of the box.
๐ Python is by far the most popular language in science, due in no small part to the ease at which it can be used and the vibrant ecosystem of user-generated packages. To install packages, there are two main methods: Pip (invoked as pip install), the package manager that comes bundled with Python, and ๐ Anaconda (invoked as conda install), a powerful package manager that can install packages for Python, R, and can download executables like Git.
Unlike R, Python was not built from the ground up with data science in mind, but there are plenty of third party libraries to make up for this. A much more exhaustive list of packages can be found later in this document, but these four packages are a good set of choices to start your data science journey with: ๐ Scikit-Learn is a general-purpose data science package which implements the most popular algorithms - it also includes rich documentation, tutorials, and examples of the models it implements. Even if you prefer to write your own implementations, Scikit-Learn is a valuable reference to the nuts-and-bolts behind many of the common algorithms you'll find. With ๐ Pandas, one can collect and analyze their data into a convenient table format. ๐ Numpy provides very fast tooling for mathematical operations, with a focus on vectors and matrices. ๐ Seaborn, itself based on the ๐ Matplotlib package, is a quick way to generate beautiful visualizations of your data, with many good defaults available out of the box, as well as a gallery showing how to produce many common visualizations of your data.
When embarking on your journey to becoming a data scientist, the choice of language isn't particularly important, and both Python and R have their pros and cons. Pick a language you like, and check out one of the Free courses we've listed below!
If you're just starting out, here's a simple recommended path:
This section contains agent frameworks and tools that are useful for data science workflows.
ย ย ย 679โญ ย ย ย 105๐ด ADK-Rust) - Production-ready AI agent development kit for Rust with model-agnostic design (Gemini, OpenAI, Anthropic), multiple agent types (LLM, Graph, Workflow), MCP support, and built-in telemetry.ย ย ย 312โญ ย ย ย ย 44๐ด Lumen) - Agent framework for chatting with data, turning natural language into SQL, transformation pipelines and visualizations. Outputs are declarative specs that can be inspected, edited, reopened in a notebook or composed into a dashboard.ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด Frostbyte MCP) - MCP server providing 13 data tools for AI agents: real-time crypto prices, IP geolocation, DNS lookups, web scraping to markdown, code execution, and screenshots. One API key for 40+ services.ย ย ย ย ย 1โญ ย ย ย ย ย 1๐ด GitHub)ย ย ย ย ย 8โญ ย ย ย ย ย 1๐ด GitHub)ย ย ย ย 46โญ ย ย ย ย 16๐ด DeepAlpha) - AI crypto trading framework using LightGBM + XGBoost ensemble with 72 ML features. 70.9% walk-forward validated accuracy on out-of-sample data. Supports Bybit and Binance. MIT licensed, available on ๐ PyPI.ย ย ย ย 21โญ ย ย ย ย ย 4๐ด CAJAL) - Local AI agent for generating publication-ready scientific papers with real arXiv citations, IMRaD structure, and tribunal scoring. Runs 100% offline via Ollama with 4B-9B models. MIT licensed. ๐ HuggingFaceย ย ย 120โญ ย ย ย ย 44๐ด ai-evaluation) - Open-source LLM and agent evaluation framework with 50+ metrics, LLM-as-Judge augmentation, and guardrail scanners (jailbreak, PII, prompt-injection). Useful for scoring RAG outputs, agent trajectories, and function-calling behavior in data-science workflows.ย ย ย 292โญ ย ย ย ย 22๐ด Kitaru) - Open-source platform that records real AI agent runs, replays them against changes, and evaluates outcomes before deployment.๐ BGPT MCP - MCP server that gives AI agents access to a database of scientific papers built from raw experimental data extracted from full-text studies. Returns 25+ structured fields per paper including methods, results, sample sizes, and quality scores. ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด GitHub)
ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด Chunk Tuner) - Open-source Python library and MCP server to benchmark document chunking strategies for RAG, score retrieval quality, and recommend configurations for a corpus.
ย ย ย ย 16โญ ย ย ย ย ย 0๐ด II-Commons) - Daily-updated skill and CLI for deterministic retrieval across arXiv, PubMed/PMC, and supported US policy corpora.
๐ Spraay x402 Gateway - x402 payment gateway with 23 Research & Reference endpoints for AI agents: Wikipedia, arXiv, PubMed, Wikidata, academic citation lookup, entity extraction, and more. Pay-per-call in USDC on Base & Solana โ no API keys or subscriptions. Also serves 150+ endpoints across 39 categories including geospatial, AI inference, DeFi, and compute. GitHub
๐ Suppr - AI literature search, document translation, and deep-research workspace for researchers.
How do you learn data science? By doing data science, of course! Okay, okay - that might not be particularly helpful when you're first starting out. In this section, we've listed some learning resources, in rough order from least to greatest commitment - Tutorials, Massively Open Online Courses (MOOCs), Intensive Programs, and Colleges.
ย ย 8406โญ ย ย 2583๐ด #tidytuesday) - A weekly data project aimed at the R ecosystem.ย ย ย 618โญ ย ย ย 253๐ด Data science your way)ย ย ย 701โญ ย ย ย 211๐ด PySpark Cheatsheet)ย ย 1260โญ ย ย ย 454๐ด Tutorials of source code from the book Genetic Algorithms with Python by Clinton Sheppard)ย ย ย ย 85โญ ย ย ย ย 27๐ด Tutorials to get started on signal processing for machine learning)ย 12822โญ ย ย 2040๐ด Minimum Viable Study Plan for Machine Learning Interviews)ย 10751โญ ย ย 1508๐ด Train LLM From Scratch) - A straightforward method for training your LLM, from downloading data to generating text.ย 21975โญ ย ย 4256๐ด Data Science) - Open Source Society Universityย 31248โญ ย ย 2585๐ด AI Expert Roadmap) - Roadmap to becoming an Artificial Intelligence Expertย ย ย 559โญ ย ย ย ย 63๐ด MLSys-NYU-2022) - Slides, scripts and materials for the Machine Learning in Finance course at NYU Tandon, 2022.ย ย ย 889โญ ย ย ย 133๐ด Hands-on Train and Deploy ML) - A hands-on course to train and deploy a serverless API that predicts crypto prices.ย ย 4156โญ ย 30923๐ด Data Science Specialization)ย ย ย 160โญ ย ย ย 195๐ด A list of colleges and universities offering degrees in data science.)This section is a collection of packages, tools, algorithms, and other useful items in the data science world.
These are some Machine Learning and Data Mining algorithms and models help you to understand your data and derive meaning from it.
ย ย ย 658โญ ย ย ย 167๐ด datacompy) - DataComPy is a package to compare two Pandas DataFrames.ย ย ย 955โญ ย ย ย 177๐ด scikit-multilearn)ย ย ย 491โญ ย ย ย ย 71๐ด sklearn-expertsys)ย ย 1586โญ ย ย ย 438๐ด scikit-feature)ย ย ย 421โญ ย ย ย ย 72๐ด scikit-rebate)ย ย ย 706โญ ย ย ย 102๐ด seqlearn)ย ย ย 520โญ ย ย ย 117๐ด sklearn-bayes)ย ย ย 440โญ ย ย ย 206๐ด sklearn-crfsuite)ย ย ย 771โญ ย ย ย 128๐ด sklearn-deap)ย ย ย ย 75โญ ย ย ย ย 11๐ด sigopt_sklearn)ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด sklearn-evaluation)ย ย 6594โญ ย ย 2414๐ด scikit-image)ย ย 6715โญ ย ย 1107๐ด scikit-opt)ย ย ย 388โญ ย ย ย ย 46๐ด scikit-posthocs)ย ย ย ย ย 1โญ ย ย ย ย ย 0๐ด me_fasttext) - Memory-efficient FastText variant with exact trie n-gram IDs, structure-aware row sharing, and mmap serving for large-vocabulary NLP.ย ย ย 667โญ ย ย ย 174๐ด pystruct)ย ย 3089โญ ย ย ย 516๐ด xLearn)ย ย 5286โญ ย ย ย 682๐ด cuML)ย ย 6005โญ ย ย ย 877๐ด causalml)ย ย 5710โญ ย ย 1720๐ด mlpack)ย ย 5175โญ ย ย ย 916๐ด MLxtend)ย ย 2365โญ ย ย ย 322๐ด modAL)ย ย 1150โญ ย ย ย 254๐ด Sparkit-learn)ย ย 2517โญ ย ย ย 168๐ด hyperlearn)ย 14445โญ ย ย 3438๐ด dlib)ย ย 1619โญ ย ย ย 142๐ด imodels)ย ย ย ย 23โญ ย ย ย ย ย 1๐ด jSciPy) - A Java port of SciPy's signal processing module, offering filters, transformations, and other scientific computing utilities.ย ย ย 443โญ ย ย ย 122๐ด RuleFit)ย ย 1015โญ ย ย ย 291๐ด pyGAM)ย ย 4056โญ ย ย ย 302๐ด Deepchecks)ย 28784โญ ย ย 8902๐ด XGBoost)ย 18794โญ ย ย 4070๐ด LightGBM)ย ย 9111โญ ย ย 1335๐ด CatBoost)ย ย ย 708โญ ย ย ย ย 42๐ด PerpetualBooster)ย 36321โญ ย ย 3794๐ด JAX)ย ย ย ย 11โญ ย ย ย ย 22๐ด PhilanthroPy) - Scikit-learn native toolkit for nonprofit fundraising analytics: leakage-safe donor propensity, lapse, planned-giving, wealth-screening and revenue-forecasting estimators.103162โญ ย 30013๐ด PyTorch)ย ย ย 215โญ ย ย ย ย 15๐ด TorchDR) - GPU and multi-GPU dimensionality reduction with a scikit-learn-compatible API.ย 17925โญ ย ย 7265๐ด torchvision)ย ย 3554โญ ย ย ย 807๐ด torchtext)ย ย 2943โญ ย ย ย 800๐ด torchaudio)ย ย 4786โญ ย ย ย 726๐ด ignite)ย ย 1723โญ ย ย ย 310๐ด PyTorchNet)ย ย ย 578โญ ย ย ย ย 63๐ด PyToune)ย ย 6179โญ ย ย ย 417๐ด skorch)ย ย ย 362โญ ย ย ย ย 51๐ด PyVarInf)ย 24098โญ ย ย 4058๐ด pytorch_geometric)ย ย 3915โญ ย ย ย 602๐ด GPyTorch)ย ย 9057โญ ย ย 1021๐ด pyro)ย ย 3385โญ ย ย ย 397๐ด Catalyst)ย ย 1687โญ ย ย ย 179๐ด pytorch_tabular)ย 10609โญ ย ย 3423๐ด Yolov3)ย 58064โญ ย 17466๐ด Yolov5)ย 61883โญ ย 11790๐ด Yolov8)200233โญ ย 77037๐ด TensorFlow)ย ย 7380โญ ย ย 1582๐ด TensorLayer)ย ย 9575โญ ย ย 2352๐ด TFLearn)ย ย 9970โญ ย ย 1305๐ด Sonnet)ย ย 6284โญ ย ย 1778๐ด tensorpack)ย ย 3130โญ ย ย ย 386๐ด TRFL)ย ย 3733โญ ย ย ย 330๐ด Polyaxon)ย ย ย 735โญ ย ย ย 158๐ด NeuPy)ย ย ย 353โญ ย ย ย ย 35๐ด tfdeploy)ย ย ย 706โญ ย ย ย 101๐ด tensorflow-upstream)ย ย 1816โญ ย ย ย 262๐ด TensorFlow Fold)ย ย ย ย 60โญ ย ย ย ย 27๐ด tensorlm)ย ย ย ย 11โญ ย ย ย ย ย 5๐ด TensorLight)ย ย 1629โญ ย ย ย 255๐ด Mesh TensorFlow)ย 11762โญ ย ย 1217๐ด Ludwig)ย ย 3027โญ ย ย ย 753๐ด TF-Agents)ย ย 3305โญ ย ย ย 523๐ด TensorForce)ย ย 1585โญ ย ย ย 635๐ด keras-contrib)ย ย 2173โญ ย ย ย 314๐ด Hyperas)ย ย 1574โญ ย ย ย 303๐ด Elephas)ย ย ย 486โญ ย ย ย ย 47๐ด Hera)ย ย 2397โญ ย ย ย 344๐ด Spektral)ย ย ย 584โญ ย ย ย 108๐ด qkeras)ย ย 5548โญ ย ย 1343๐ด keras-rl)ย ย 1636โญ ย ย ย 264๐ด Talos)ย 33506โญ ย ย 3184๐ด Netron)ย ย ย ย ย 7โญ ย ย ย ย ย 0๐ด Resseract Lite)ย ย 2036โญ ย ย ย ย 87๐ด vizzu)ย ย 3471โญ ย ย ย 360๐ด TensorWatch)ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด github.com)ย ย ย 765โญ ย ย ย ย 32๐ด torchvista) - Interactive notebook-based tool to visualize the forward pass of any PyTorch model.ย ย ย ย 25โญ ย ย ย ย ย 2๐ด FlexViz) - Python library for interactive, cross-filtered dashboards that stay responsive on 100M+ rows by aggregating with Polars on the server.| Link | Description |
|---|---|
ย ย ย 529โญ ย ย ย ย 75๐ด The Data Science Lifecycle Process) | The Data Science Lifecycle Process is a process for taking data science teams from Idea to Value repeatedly and sustainably. The process is documented in this repo |
ย ย ย 205โญ ย ย ย ย 60๐ด Data Science Lifecycle Template Repo) | Template repository for data science lifecycle project |
ย ย ย 570โญ ย ย ย ย 82๐ด TabGAN) | Synthetic tabular data generation using GANs, Diffusion Models, and LLMs with adversarial filtering and privacy metrics. |
ย ย ย 277โญ ย ย ย ย 24๐ด RexMex) | A general purpose recommender metrics library for fair evaluation. |
ย ย ย 785โญ ย ย ย 100๐ด ChemicalX) | A PyTorch based deep learning library for drug pair scoring. |
ย ย ย ย 30โญ ย ย ย ย ย 5๐ด FileShot.io) | Secure zero-knowledge encrypted file sharing (AES-256-GCM in-browser). No account required, MIT licensed, self-hostable, optional link expiry. |
| ๐ CorpusExplorer | Software for corpus linguists and text/data mining enthusiasts. Build your own corpora in over 60 languages. Use over 50 tools/visualizations. |
ย ย 2990โญ ย ย ย 402๐ด PyTorch Geometric Temporal) | Representation learning on dynamic graphs. |
ย ย ย 715โญ ย ย ย ย 54๐ด Little Ball of Fur) | A graph sampling library for NetworkX with a Scikit-Learn like API. |
ย ย 2286โญ ย ย ย 254๐ด Karate Club) | An unsupervised machine learning extension library for NetworkX with a Scikit-Learn like API. |
ย ย 3546โญ ย ย ย 456๐ด ML Workspace) | All-in-one web-based IDE for machine learning and data science. The workspace is deployed as a Docker container and is preloaded with a variety of popular data science libraries (e.g., Tensorflow, PyTorch) and dev tools (e.g., Jupyter, VS Code) |
ย ย 9649โญ ย ย ย 742๐ด xonsh shell) | A Python-powered shell that enables integration, management and orchestration of data science libraries mostly written in Python, allowing you to build pipelines, code and command-based workflows. It can also be used as a kernel for Jupyter Notebook. |
| ๐ Neptune.ai | Community-friendly platform supporting data scientists in creating and sharing machine learning models. Neptune facilitates teamwork, infrastructure management, models comparison and reproducibility. |
ย ย ย 136โญ ย ย ย ย 32๐ด steppy) | Lightweight, Python library for fast and reproducible machine learning experimentation. Introduces very simple interface that enables clean machine learning pipeline design. |
ย ย ย ย 23โญ ย ย ย ย ย 9๐ด steppy-toolkit) | Curated collection of the neural networks, transformers and models that make your machine learning work faster and more effective. |
| ๐ Datalab from Google | easily explore, visualize, analyze, and transform data using familiar languages, such as Python and SQL, interactively. |
| ๐ Hortonworks Sandbox | is a personal, portable Hadoop environment that comes with a dozen interactive Hadoop tutorials. |
| ๐ R | is a free software environment for statistical computing and graphics. |
| ๐ Tidyverse | is an opinionated collection of R packages designed for data science. All packages share an underlying design philosophy, grammar, and data structures. |
| ๐ RStudio | IDE โ powerful user interface for R. Itโs free and open source, and works on Windows, Mac, and Linux. |
| ๐ Python - Pandas - Anaconda | Completely free enterprise-ready Python distribution for large-scale data processing, predictive analytics, and scientific computing |
ย ย 3256โญ ย ย ย 239๐ด Pandas GUI) | Pandas GUI |
ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด NuriStat) | Free open-source SPSS alternative โ menu-driven desktop statistics (t-tests, ANOVA, regression, survival analysis, ROC) with SPSS .sav import/export |
ย 39834โญ ย ย 3121๐ด Polars) | Fast DataFrame library for Rust and Python, designed as a faster alternative to Pandas |
| ๐ CiteMe | free academic citation generator with a built-in reference checker that flags fabricated or hallucinated references. Searches 11+ scholarly databases (OpenAlex, PubMed, Semantic Scholar, CrossRef, SciELO), formats 40+ citation styles, and offers a public API. No sign-up; available in English, Spanish, Portuguese, French, and German. |
| ๐ Scikit-Learn | Machine Learning in Python |
| ๐ NumPy | NumPy is fundamental for scientific computing with Python. It supports large, multi-dimensional arrays and matrices and includes an assortment of high-level mathematical functions to operate on these arrays. |
| ๐ Vaex | Vaex is a Python library that allows you to visualize large datasets and calculate statistics at high speeds. |
| ๐ SciPy | SciPy works with NumPy arrays and provides efficient routines for numerical integration and optimization. |
| ๐ Data Science Toolbox | Coursera Course |
| ๐ Data Science Toolbox | Blog |
| ๐ Wolfram Data Science Platform | Take numerical, textual, image, GIS or other data and give it the Wolfram treatment, carrying out a full spectrum of data science analysis and visualization and automatically generate rich interactive reportsโall powered by the revolutionary knowledge-based Wolfram Language. |
| ๐ Datadog | Solutions, code, and devops for high-scale data science. |
| ๐ Variance | Build powerful data visualizations for the web without writing JavaScript |
| ๐ Kite Development Kit | The Kite Software Development Kit (Apache License, Version 2.0), or Kite for short, is a set of libraries, tools, examples, and documentation focused on making it easier to build systems on top of the Hadoop ecosystem. |
| ๐ Domino Data Labs | Run, scale, share, and deploy your models โ without any infrastructure or setup. |
| ๐ Apache Flink | A platform for efficient, distributed, general-purpose data processing. |
| ๐ Apache Hama | Apache Hama is an Apache Top-Level open source project, allowing you to do advanced analytics beyond MapReduce. |
| ๐ Weka | Weka is a collection of machine learning algorithms for data mining tasks. |
| ๐ Octave | GNU Octave is a high-level interpreted language, primarily intended for numerical computations.(Free Matlab) |
| ๐ Apache Spark | Lightning-fast cluster computing |
ย ย ย 326โญ ย ย ย ย 69๐ด Hydrosphere Mist) | a service for exposing Apache Spark analytics jobs and machine learning models as realtime, batch or reactive web services. |
| ๐ Data Mechanics | A data science and engineering platform making Apache Spark more developer-friendly and cost-effective. |
| ๐ Caffe | Deep Learning Framework |
| ๐ Torch | A SCIENTIFIC COMPUTING FRAMEWORK FOR LUAJIT |
ย ย 3860โญ ย ย ย 804๐ด Nervana's python based Deep Learning Framework) | Intelยฎ Nervanaโข reference deep learning framework committed to best performance on all hardware. |
ย ย ย 396โญ ย ย ย ย 52๐ด Skale) | High performance distributed data processing in NodeJS |
| ๐ Aerosolve | A machine learning package built for humans. |
ย ย ย 312โญ ย ย ย ย 80๐ด Intel framework) | Intelยฎ Deep Learning Framework |
| ๐ Datawrapper | An open source data visualization platform helping everyone to create simple, correct and embeddable charts. Also at ย ย 1458โญ ย ย ย 279๐ด github.com) |
| ๐ Tensor Flow | TensorFlow is an Open Source Software Library for Machine Intelligence |
| ๐ Natural Language Toolkit | An introductory yet powerful toolkit for natural language processing and classification |
ย 20466โญ ย ย 2043๐ด FunASR) | Industrial-grade speech recognition toolkit supporting 50+ languages with built-in VAD, punctuation, speaker diarization, and emotion detection. OpenAI-compatible API server included. |
| ๐ Annotation Lab | Free End-to-End No-Code platform for text annotation and DL model training/tuning. Out-of-the-box support for Named Entity Recognition, Classification, Relation extraction and Assertion Status Spark NLP models. Unlimited support for users, teams, projects, documents. |
| ๐ nlp-toolkit for node.js | This module covers some basic nlp principles and implementations. The main focus is performance. When we deal with sample or training data in nlp, we quickly run out of memory. Therefore every implementation in this module is written as stream to only hold that data in memory that is currently processed at any step. |
| ๐ Julia | high-level, high-performance dynamic programming language for technical computing |
ย ย 2905โญ ย ย ย 426๐ด IJulia) | a Julia-language backend combined with the Jupyter interactive environment |
| ๐ Apache Zeppelin | Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more |
ย ย 7683โญ ย ย ย 915๐ด Featuretools) | An open source framework for automated feature engineering written in python |
ย ย 1536โญ ย ย ย 229๐ด Optimus) | Cleansing, pre-processing, feature engineering, exploratory data analysis and easy ML with PySpark backend. |
ย 15309โญ ย ย 1706๐ด Albumentations) | ะ fast and framework agnostic image augmentation library that implements a diverse set of augmentation techniques. Supports classification, segmentation, and detection out of the box. Was used to win a number of Deep Learning competitions at Kaggle, Topcoder and those that were a part of the CVPR workshops. |
ย 15881โญ ย ย 1329๐ด DVC) | An open-source data science version control system. It helps track, organize and make data science projects reproducible. In its very basic scenario it helps version control and share large data and model files. |
ย ย ย ย 26โญ ย ย ย ย ย 5๐ด Lambdo) | is a workflow engine that significantly simplifies data analysis by combining in one analysis pipeline (i) feature engineering and machine learning (ii) model training and prediction (iii) table population and column evaluation. |
ย ย 7301โญ ย ย 1442๐ด Feast) | A feature store for the management, discovery, and access of machine learning features. Feast provides a consistent view of feature data for both model training and model serving. |
ย ย 3733โญ ย ย ย 330๐ด Polyaxon) | A platform for reproducible and scalable machine learning and deep learning. |
| ๐ UBIAI | Easy-to-use text annotation tool for teams with most comprehensive auto-annotation features. Supports NER, relations and document classification as well as OCR annotation for invoice labeling |
ย ย 6883โญ ย ย ย 801๐ด Trains) | Auto-Magical Experiment Manager, Version Control & DevOps for AI |
ย ย 1307โญ ย ย ย 159๐ด Hopsworks) | Open-source data-intensive machine learning platform with a feature store. Ingest and manage features for both online (MySQL Cluster) and offline (Apache Hive) access, train and serve models at scale. |
ย 39763โญ ย ย 6244๐ด MindsDB) | MindsDB is an Explainable AutoML framework for developers. With MindsDB you can build, train and use state of the art ML models in as simple as one line of code. |
ย ย ย 510โญ ย ย ย 101๐ด Lightwood) | A Pytorch based framework that breaks down machine learning problems into smaller blocks that can be glued together seamlessly with an objective to build predictive models with one line of code. |
ย ย 4119โญ ย ย ย 749๐ด AWS Data Wrangler) | An open-source Python package that extends the power of Pandas library to AWS connecting DataFrames and AWS data related services (Amazon Redshift, AWS Glue, Amazon Athena, Amazon EMR, etc). |
| ๐ Amazon Rekognition | AWS Rekognition is a service that lets developers working with Amazon Web Services add image analysis to their applications. Catalog assets, automate workflows, and extract meaning from your media and applications. |
| ๐ Amazon Textract | Automatically extract printed text, handwriting, and data from any document. |
| ๐ Amazon Lookout for Vision | Spot product defects using computer vision to automate quality inspection. Identify missing product components, vehicle and structure damage, and irregularities for comprehensive quality control. |
| ๐ Amazon CodeGuru | Automate code reviews and optimize application performance with ML-powered recommendations. |
ย ย 4187โญ ย ย ย 345๐ด CML) | An open source toolkit for using continuous integration in data science projects. Automatically train and test models in production-like environments with GitHub Actions & GitLab CI, and autogenerate visual reports on pull/merge requests. |
| ๐ Dask | An open source Python library to painlessly transition your analytics code to distributed computing systems (Big Data) |
ย 41617โญ ย ย 3814๐ด DuckDB) | An in-process SQL OLAP database management system |
| ๐ Statsmodels | A Python-based inferential statistics, hypothesis testing and regression framework |
| ๐ Gensim | An open-source library for topic modeling of natural language text |
| ๐ spaCy | A performant natural language processing toolkit |
ย ย 8809โญ ย ย 1477๐ด Grid Studio) | Grid studio is a web-based spreadsheet application with full integration of the Python programming language. |
ย 49968โญ ย 19121๐ด Python Data Science Handbook) | Python Data Science Handbook: full text in Jupyter Notebooks |
ย ย ย 228โญ ย ย ย ย 34๐ด Shapley) | A data-driven framework to quantify the value of classifiers in a machine learning ensemble. |
| ๐ DAGsHub | A platform built on open source tools for data, model and pipeline management. |
| ๐ Deepnote | A new kind of data science notebook. Jupyter-compatible, with real-time collaboration and running in the cloud. |
| ๐ Valohai | An MLOps platform that handles machine orchestration, automatic reproducibility and deployment. |
| ๐ PyMC3 | A Python Library for Probabalistic Programming (Bayesian Inference and Machine Learning) |
| ๐ PyStan | Python interface to Stan (Bayesian inference and modeling) |
| ๐ hmmlearn | Unsupervised learning and inference of Hidden Markov Models |
ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด Chaos Genius) | ML powered analytics engine for outlier/anomaly detection and root cause analysis |
| ๐ Nimblebox | A full-stack MLOps platform designed to help data scientists and machine learning practitioners around the world discover, create, and launch multi-cloud apps from their web browser. |
ย ย 3450โญ ย ย ย 257๐ด Towhee) | A Python library that helps you encode your unstructured data into embeddings. |
ย ย ย 672โญ ย ย ย ย 58๐ด LineaPy) | Ever been frustrated with cleaning up long, messy Jupyter notebooks? With LineaPy, an open source Python library, it takes as little as two lines of code to transform messy development code into production pipelines. |
ย ย 2233โญ ย ย ย 168๐ด envd) | ๐๏ธ machine learning development environment for data science and AI/ML engineering teams |
| ๐ Explore Data Science Libraries | A search engine ๐ tool to discover & find a curated list of popular & new libraries, top authors, trending project kits, discussions, tutorials & learning resources |
ย ย ย 718โญ ย ย ย ย 42๐ด MLEM) | ๐ถ Version and deploy your ML models following GitOps principles |
| ๐ MLflow | MLOps framework for managing ML models across their full lifecycle |
ย 11669โญ ย ย ย 919๐ด cleanlab) | Python library for data-centric AI and automatically detecting various issues in ML datasets |
ย 10728โญ ย ย 1195๐ด AutoGluon) | AutoML to easily produce accurate predictions for image, text, tabular, time-series, and multi-modal data |
| ๐ Arize AI | Arize AI community tier observability tool for monitoring machine learning models in production and root-causing issues such as data quality and performance drift. |
| ๐ Aureo.io | Aureo.io is a low-code platform that focuses on building artificial intelligence. It provides users with the capability to create pipelines, automations and integrate them with artificial intelligence models โ all with their basic data. |
| ๐ ERD Lab | Free cloud based entity relationship diagram (ERD) tool made for developers. |
| ๐ Arize-Phoenix | MLOps in a notebook - uncover insights, surface problems, monitor, and fine tune your models. |
ย ย ย 176โญ ย ย ย ย 70๐ด Comet) | An MLOps platform with experiment tracking, model production management, a model registry, and full data lineage to support your ML workflow from training straight through to production. |
ย 22188โญ ย ย 1817๐ด Opik) | Evaluate, test, and ship LLM applications across your dev and production lifecycles. |
| ๐ Synthical | AI-powered collaborative environment for research. Find relevant papers, create collections to manage bibliography, and summarize content โ all in one place |
ย ย ย ย 13โญ ย ย ย ย ย 0๐ด teeplot) | Workflow tool to automatically organize data visualization output |
ย 45814โญ ย ย 4387๐ด Streamlit) | App framework for Machine Learning and Data Science projects |
ย 43596โญ ย ย 3608๐ด Gradio) | Create customizable UI components around machine learning models |
ย 11258โญ ย ย ย 900๐ด Weights & Biases) | Experiment tracking, dataset versioning, and model management |
ย 15881โญ ย ย 1329๐ด DVC) | Open-source version control system for machine learning projects |
ย 14830โญ ย ย 1391๐ด Optuna) | Automatic hyperparameter optimization software framework |
ย 43889โญ ย ย 8068๐ด Ray Tune) | Scalable hyperparameter tuning library |
ย 46932โญ ย 17892๐ด Apache Airflow) | Platform to programmatically author, schedule, and monitor workflows |
ย 23893โญ ย ย 2534๐ด Prefect) | Workflow management system for modern data stacks |
ย 11006โญ ย ย 1079๐ด Kedro) | Open-source Python framework for creating reproducible, maintainable data science code |
ย ย 2595โญ ย ย ย 216๐ด Hamilton) | Lightweight library to author and manage reliable data transformations |
ย 25772โญ ย ย 3750๐ด SHAP) | Game theoretic approach to explain the output of any machine learning model |
ย ย 6946โญ ย ย ย 787๐ด InterpretML) | InterpretML implements the Explainable Boosting Machine (EBM), a modern, fully interpretable machine learning model based on Generalized Additive Models (GAMs). This open-source package also provides visualization tools for EBMs, other glass-box models, and black-box explanations |
ย 12163โญ ย ย 1846๐ด LIME) | Explaining the predictions of any machine learning classifier |
ย ย 7547โญ ย ย ย 890๐ด flyte) | Workflow automation platform for machine learning |
ย 13899โญ ย ย 2574๐ด dbt) | Data build tool |
ย ย 2342โญ ย ย ย ย 75๐ด zasper) | Supercharged IDE for Data Science |
ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด skrub) | A Python library to ease preprocessing and feature engineering for tabular machine learning |
ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด Glyph) | Framework-agnostic TypeScript library for generating, searching, and comparing MinHash fingerprints for fast text similarity, deduplication, and retrieval. |
| ๐ Codeflash | Ship Blazing-Fast Python Code โ Every Time |
| ๐ Hugging Face | Popular open platform for sharing ML models, datasets, and collaborating on NLP and generative AI projects. |
ย ย ย ย 73โญ ย ย ย ย ย 6๐ด Chinese-Elite) | An open-source project that automatically maps relationship networks by parsing public data using LLMs and visualizes it as an interactive graph. |
ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด Desbordante) | An open-source data profiler specifically focused on discovery and validation of complex patterns, such as ๐ numerical association rules, ๐ differential dependencies, ๐ denial constraints, and more. |
ย ย ย ย 57โญ ย ย ย ย ย 8๐ด dna-claude-analysis) | Personal genome analysis toolkit with Python scripts analyzing raw DNA data across 17 categories (health risks, ancestry, pharmacogenomics, nutrition, psychology, and more) and generating a terminal-style single-page HTML visualization. |
ย ย ย 257โญ ย ย ย ย 10๐ด RunMat) | Fast MATLAB-syntax runtime with automatic CPU/GPU execution and fused array kernels. |
ย ย ย ย 17โญ ย ย ย ย ย 2๐ด Turbostream) | A terminal UI for experimenting with custom rule engines and selective LLM analysis on real-time data streams, without worrying about streaming infra or backpressure. |
ย ย 1792โญ ย ย ย 166๐ด WFGY ProblemMap) | Open source โfailure atlasโ of 16 recurring issues in LLM and RAG pipelines, with observable symptoms and suggested fixes for data science teams. |
| ๐ Deploybase | Track real-time GPU and LLM pricing across all cloud and inference providers. |
ย ย 4647โญ ย ย ย 736๐ด DeepAnalyze) | An agentic LLM for autonomous data science, which can autonomously complete a wide range of data science tasks without human intervention. |
ย ย ย ย ย 7โญ ย ย ย ย ย 0๐ด Disco) | Superhuman exploratory data analysis. Finds the feature interactions and subgroup effects in tabular data that LLMs and manual exploration miss โ with p-values, effect sizes, and literature citations. Free for public data. |
| ๐ AI for Database | Chat with your database in natural language โ no SQL needed. Get instant insights, build self-refreshing dashboards, and trigger automated workflows based on database changes. |
ย ย ย ย 46โญ ย ย ย ย 16๐ด Crypto Pump Scanner) | AI-powered cryptocurrency trading bot with LSTM neural network (84.6% accuracy). Real-time pump detection, walk-forward validated models, multi-exchange support (Bybit, Binance, OKX, Gate.io). Open source. |
ย ย 2050โญ ย ย ย 632๐ด Future AGI) | Open-source platform to simulate, evaluate, trace, guardrail, route, and optimize LLM and AI agent apps in one feedback loop, so agents don't just get monitored, they self-improve. Self-hostable. Apache-2.0. |
This section includes some additional reading material, channels to watch, and talks to listen to.
๐ Managing ML Projects
๐ Data for All
ย ย 6761โญ 241810๐ด How to Share Data with a Statistician)Below are some Social Media links. Connect with other data scientists!
| Description | |
|---|---|
| ๐ Big Data Combine | Rapid-fire, live tryouts for data scientists seeking to monetize their models as trading strategies |
| Big Data Mania | Data Viz Wiz, Data Journalist, Growth Hacker, Author of Data Science for Dummies (2015) |
| ๐ Big Data Science | Big Data, Data Science, Predictive Modeling, Business Analytics, Hadoop, Decision and Operations Research. |
| Charlie Greenbacker | Director of Data Science at @ExploreAltamira |
| ๐ Chris Said | Data scientist at Twitter |
| ๐ Clare Corthell | Dev, Design, Data Science @mattermark #hackerei |
| ๐ DADI Charles-Abner | #datascientist @Ekimetrics. , #machinelearning #dataviz #DynamicCharts #Hadoop #R #Python #NLP #Bitcoin #dataenthousiast |
| ๐ Data Science Central | Data Science Central is the industry's single resource for Big Data practitioners. |
| ๐ Data Science London | Data Science. Big Data. Data Hacks. Data Junkies. Data Startups. Open Data |
| ๐ Data Science Renee | Documenting my path from SQL Data Analyst pursuing an Engineering Master's Degree to Data Scientist |
| ๐ Data Science Report | Mission is to help guide & advance careers in Data Science & Analytics |
| ๐ Data Science Tips | Tips and Tricks for Data Scientists around the world! #datascience #bigdata |
| ๐ Data Vizzard | DataViz, Security, Military |
| ๐ DataScienceX | |
| deeplearning4j | |
| ๐ DJ Patil | White House Data Chief, VP @ RelateIQ. |
| ๐ Domino Data Lab | |
| ๐ Drew Conway | Data nerd, hacker, student of conflict. |
| Emilio Ferrara | #Networks, #MachineLearning and #DataScience. I work on #Social Media. Postdoc at @IndianaUniv |
| ๐ Erin Bartolo | Running with #BigData--enjoying a love/hate relationship with its hype. @iSchoolSU #DataScience Program Mgr. |
| ๐ Greg Reda | Working @ GrubHub about data and pandas |
| ๐ Gregory Piatetsky | KDnuggets President, Analytics/Big Data/Data Mining/Data Science expert, KDD & SIGKDD co-founder, was Chief Scientist at 2 startups, part-time philosopher. |
| ๐ Hadley Wickham | Chief Scientist at RStudio, and an Adjunct Professor of Statistics at the University of Auckland, Stanford University, and Rice University. |
| ๐ Hakan Kardas | Data Scientist |
| ๐ Hilary Mason | Data Scientist in Residence at @accel. |
| ๐ Jeff Hammerbacher | ReTweeting about data science |
| ๐ John Myles White | Scientist at Facebook and Julia developer. Author of Machine Learning for Hackers and Bandit Algorithms for Website Optimization. Tweets reflect my views only. |
| ๐ Juan Miguel Lavista | Principal Data Scientist @ Microsoft Data Science Team |
| ๐ Julia Evans | Hacker - Pandas - Data Analyze |
| ๐ Kenneth Cukier | The Economist's Data Editor and co-author of Big Data (https://www.big-data-book.com/). |
| Kevin Davenport | Organizer of https://www.meetup.com/San-Diego-Data-Science-R-Users-Group/ |
| ๐ Kevin Markham | Data science instructor, and founder of ๐ Data School |
| ๐ Kim Rees | Interactive data visualization and tools. Data flaneur. |
| ๐ Kirk Borne | DataScientist, PhD Astrophysicist, Top #BigData Influencer. |
| Linda Regber | Data storyteller, visualizations. |
| ๐ Luis Rei | PhD Student. Programming, Mobile, Web. Artificial Intelligence, Intelligent Robotics Machine Learning, Data Mining, Natural Language Processing, Data Science. |
| Mark Stevenson | Data Analytics Recruitment Specialist at Salt (@SaltJobs) Analytics - Insight - Big Data - Data science |
| ๐ Matt Harrison | Opinions of full-stack Python guy, author, instructor, currently playing Data Scientist. Occasional fathering, husbanding, organic gardening. |
| ๐ Matthew Russell | Mining the Social Web. |
| ๐ Mert Nuhoฤlu | Data Scientist at BizQualify, Developer |
| ๐ Monica Rogati | Data @ Jawbone. Turned data into stories & products at LinkedIn. Text mining, applied machine learning, recommender systems. Ex-gamer, ex-machine coder; namer. |
| ๐ Noah Iliinsky | Visualization & interaction designer. Practical cyclist. Author of vis books: https://www.oreilly.com/pub/au/4419 |
| ๐ Paul Miller | Cloud Computing/ Big Data/ Open Data Analyst & Consultant. Writer, Speaker & Moderator. Gigaom Research Analyst. |
| ๐ Peter Skomoroch | Creating intelligent systems to automate tasks & improve decisions. Entrepreneur, ex-Principal Data Scientist @LinkedIn. Machine Learning, ProductRei, Networks |
| ๐ Prash Chan | Solution Architect @ IBM, Master Data Management, Data Quality & Data Governance Blogger. Data Science, Hadoop, Big Data & Cloud. |
| ๐ Quora Data Science | Quora's data science topic |
| ๐ R-Bloggers | Tweet blog posts from the R blogosphere, data science conferences, and (!) open jobs for data scientists. |
| ๐ Rand Hindi | |
| ๐ Randy Olson | Computer scientist researching artificial intelligence. Data tinkerer. Community leader for @DataIsBeautiful. #OpenScience advocate. |
| ๐ Recep Erol | Data Science geek @ UALR |
| ๐ Ryan Orban | Data scientist, genetic origamist, hardware aficionado |
| ๐ Sean J. Taylor | Social Scientist. Hacker. Facebook Data Science Team. Keywords: Experiments, Causal Inference, Statistics, Machine Learning, Economics. |
| ๐ Silvia K. Spiva | #DataScience at Cisco |
| ๐ Harsh B. Gupta | Data Scientist at BBVA Compass |
| ๐ Spencer Nelson | Data nerd |
| ๐ Talha Oz | Enjoys ABM, SNA, DM, ML, NLP, HI, Python, Java. Top percentile Kaggler/data scientist |
| ๐ Tasos Skarlatidis | Complex Event Processing, Big Data, Artificial Intelligence and Machine Learning. Passionate about programming and open-source. |
| ๐ Terry Timko | InfoGov; Bigdata; Data as a Service; Data Science; Open, Social & Business Data Convergence |
| ๐ Tony Baer | IT analyst with Ovum covering Big Data & data management with some systems engineering thrown in. |
| ๐ Tony Ojeda | Data Scientist , Author , Entrepreneur. Co-founder @DataCommunityDC. Founder @DistrictDataLab. #DataScience #BigData #DataDC |
| ๐ Vamshi Ambati | Data Science @ PayPal. #NLP, #machinelearning; PhD, Carnegie Mellon alumni (Blog: https://allthingsds.wordpress.com ) |
| ๐ Wes McKinney | Pandas (Python Data Analysis library). |
| ๐ WileyEd | Senior Manager - @Seagate Big Data Analytics @McKinsey Alum #BigData + #Analytics Evangelist #Hadoop, #Cloud, #Digital, & #R Enthusiast |
| ๐ WNYC Data News Team | The data news crew at @WNYC. Practicing data-driven journalism, making it visual, and showing our work. |
| ๐ Alexey Grigorev | Data science author |
| ๐ ฤฐlker Arslan | Data science author. Shares mostly about Julia programming |
| ๐ INEVITABLE | AI & Data Science Start-up Company based in England, UK |
| ๐ Jan Oliver Rรผdiger | ML, DL and Data Science - with a focus on text-/data-mining |
Some data mining competition platforms
| Preview | Description |
|---|---|
๐ ![]() | ๐ Key differences of a data scientist vs. data engineer |
๐ ![]() | A visual guide to Becoming a Data Scientist in 8 Steps by ๐ DataCamp ๐ (img) |
๐ ![]() | Mindmap on required skills ๐ img) |
๐ ![]() | Swami Chandrasekaran made a ๐ Curriculum via Metro map. |
๐ ![]() | by ๐ @kzawadz via ๐ twitter |
๐ ![]() | By ๐ Data Science Central |
๐ ![]() | Data Science Wars: R vs Python |
๐ ![]() | How to select statistical or machine learning techniques |
| ๐ | ๐ Choosing the Right Estimator |
๐ ![]() | The Data Science Industry: Who Does What |
๐ ![]() | Data Science |
๐ ![]() | Different Data Science Skills and Roles from ๐ Springboard |
๐ ![]() | A simple and friendly way of teaching your non-data scientist/non-statistician colleagues ๐ how to avoid mistakes with data. From Geckoboard's ๐ Data Literacy Lessons. |
ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด College ROI Dataset) - Lifetime return-on-investment estimates for ~30K US bachelor's programs across 1,775 institutions, built from FREOPP, IPEDS, and BEA regional price data. 5 CSVs with data dictionary, CC BY 4.0, Zenodo DOI.ย ย ย ย ย ?โญ ย ย ย ย ย ?๐ด AI Displacement Tracker) - Structured dataset tracking 92 AI-attributed workforce reduction events affecting 453,748 workers across 12 countries and 11 sectors. JSON and CSV formats. CC-BY-4.0 licensed.ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด Pokemon Card Centering Measurements) - 320 measured PSA-style centering annotations (left/right and top/bottom border percentages, tilt) across 302 real eBay-listed Pokemon cards. CSV, CC BY 4.0, Zenodo DOI.ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด Pokemon Card Sold-Price Reference by Grade) - Median sold price by grade (raw, PSA 9, PSA 10) for 486 Pokemon cards, with sample size and confidence flag per card. CSV, CC BY 4.0, Zenodo DOI.ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด GitHub), and HuggingFace mirror. Permanent DOI, CC BY 4.0.ย ย ย 523โญ ย ย ย 192๐ด Open Data Sources)ย ย ย 349โญ ย ย ย ย 82๐ด Public Git Archive)ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด NAYN.CO Turkish News with categories)ย ย 1166โญ ย ย ย 596๐ด Covid-19)ย ย ย 117โญ ย ย ย ย 68๐ด Covid-19 Google)ย ย ย 119โญ ย ย ย ย 43๐ด 5000 Images of Clothes)ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด HDD Price Index) - Daily open dataset of the cheapest new internal 3.5" SATA hard-drive price per terabyte (USD/TB) by capacity tier on Amazon US, with a historical time series. CSV, JSON and JSONL, no login, CC BY 4.0.ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด BDE Score) - AI-powered multi-market stock analysis with transparent BDE scoring across 73 stocks (US/HK/A-share). EU AI Act Art.50 compliant. MIT license.ย ย ย 183โญ ย ย ย ย 33๐ด FirstData) - The world's most comprehensive authoritative data source knowledge base. 210+ curated sources from governments, international organizations, and research institutions. MCP integration for AI agents. MIT licensed.ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด latamdata-py) - Python package for one-line access to 38 open research datasets from Latin America (health, neuroscience, mental health, economics). pip install latamdata-py.ย ย ย ย ย 4โญ ย ย ย ย ย 0๐ด ZipCheckup) - Free ZIP-level environmental safety data for 42,000+ US ZIP codes: water quality, air quality, PFAS contamination, radon, lead, flood risk, and 11 more verticals. Public REST API, npm/PyPI packages, CC BY 4.0.ย ย ย ย ย 3โญ ย ย ย ย ย 0๐ด Verified Supplement Evidence) - Evidence-graded dietary-supplement dataset covering dosing, bioavailability by form, drug-nutrient interactions, NHANES deficiency prevalence, FDA FAERS adverse-event signals, and cost-per-effective-dose, with every clinical claim citing a PubMed PMID. CC BY 4.0, DOI 10.57967/hf/9356.ย ย ย ย ย 0โญ ย ย ย ย ย 0๐ด WhatFontIs-Bench) - Synthetic benchmark for font family identification with 11,995 images of words set in 600 known fonts, annotated with word and per-letter boxes.ย 33679โญ ย ย 3602๐ด awesome-awesomeness)ย 74391โญ ย 15651๐ด Awesome Machine Learning)ย 11504โญ ย ย ย 785๐ด lists)ย ย 4414โญ ย ย ย 471๐ด awesome-dataviz)322202โญ ย 28776๐ด awesome-python)ย 29354โญ ย ย 8016๐ด Data Science IPython Notebooks.)ย ย 6514โญ ย ย 1513๐ด awesome-r)ย 79094โญ ย 11848๐ด awesome-datasets)ย 18216โญ ย ย 3966๐ด awesome-Machine Learning & Deep Learning Tutorials)ย ย ย 693โญ ย ย ย ย 89๐ด Awesome Data Science Ideas)ย 28866โญ ย ย 6144๐ด Machine Learning for Software Engineers)ย ย 6639โญ ย ย ย 830๐ด Awesome Machine Learning On Source Code)ย ย 2451โญ ย ย ย 356๐ด Awesome Community Detection)ย ย 4801โญ ย ย ย 723๐ด Awesome Graph Classification)ย ย 2475โญ ย ย ย 341๐ด Awesome Decision Tree Papers)ย ย 1834โญ ย ย ย 329๐ด Awesome Fraud Detection Papers)ย ย 1051โญ ย ย ย 166๐ด Awesome Gradient Boosting Papers)ย ย ย 543โญ ย ย ย ย 94๐ด Awesome Computer Vision Models)ย ย ย 715โญ ย ย ย ย 76๐ด Awesome Monte Carlo Tree Search)ย ย 3851โญ ย ย ย 559๐ด 100 NLP Papers)ย ย 1124โญ ย ย ย ย 87๐ด Awesome Game Datasets)ย ย ย ย ย 2โญ ย ย ย ย ย 0๐ด ML/AI Interview Prep) - 500+ ML/AI interview Q&A with runnable code โ covers ML fundamentals, deep learning, NLP, PyTorch, scikit-learn pipelines, and system designย 10154โญ ย ย 2154๐ด Data Science Interviews Questions)ย ย 1990โญ ย ย ย 136๐ด Awesome Explainable Graph Reasoning)ย ย ย 101โญ ย ย ย ย 15๐ด Awesome Drug Synergy, Interaction and Polypharmacy Prediction)ย ย 2788โญ ย ย ย 645๐ด Data Science Projects)ย ย 1975โญ ย ย ย 284๐ด Awesome Data Analysis) - A curated list of data analysis tools, libraries and resources.ย ย ย ย 27โญ ย ย ย ย ย 9๐ด Awesome Evidence Synthesis) - A curated list of open-source tools for systematic reviews, meta-analysis, and evidence synthesis.ย ย 1502โญ ย ย ย 160๐ด Awesome Music Production)ย 30046โญ ย ย 6641๐ด academic/awesome-datascience)