Splunk App for Data Science and Deep Learning - container images repository
64
stars
365
commits
Jupyter Notebook
primary language
Sep 9, 2026
updated
README
Splunk App for Data Science and Deep Learning
Splunk App for Data Science and Deep Learning (DSDL) 5.2.4 formerly known as Deep Learning Toolkit for Splunk (DLTK) versions 2.3.0 - 3.9.0 and (DSDL) 5.0.0 - 5.2.4 published on splunkbase. There is also a prior DLTK version 4.x which was open sourced on GitHub but is not actively maintained. Please make use of the latest version 5.2.4
Copyright (C) 2005-2026 Splunk Inc. All rights reserved.
Author: Philipp Drieger
Contributors: Josh Cowling, Huaibo Zhao, Tatsu Murata
About
This repository contains the collection of resources, scripts, and testing frameworks that are used build and deploy the default container images used by the DSDL app. It can be used to modify, secure, update and change these containers and their build process to meet the needs of your enterprise environment.
Resources
This repository contains the container endpoint (./app), jupyter notebook configuration (./config) and examples (./notebooks), build scripts and the main Dockerfiles to create the existing pre-built container images for TensorFlow, PyTorch, NLP libraries and many other data science libraries for CPU and GPU.
Building containers
You can build your own containers with the build.sh script.
The build.sh script is invoked with at least one and up to three arguments:
./build.sh <build_configuration_tag> <repo_name> <version>
<build_configuration_tag> is used to specify the particular set of base docker image, dockerfile, base python requirements, specific python requirements and runtime setting.
These combinations can be found and added to in tag_mapping.csv.
<repo_name> allows you to specify a repo prefix which will be needed if you intend to upload images to dockerhub.
<version> allows you to specify a new version number for the completed image.
To build the default golden cpu image locally, simply run:
./build.sh golden-cpu
or specify additional arguments:
./build.sh golden-cpu local_build/ 1.0.0
For the current v5.2.4 release, the following tags are available in tag_mapping.csv. You can still add custom entries for your own builds:
build configuration tag | Description |
|---|---|
| golden-cpu | Red Hat UBI Golden CPU (5.2.4) |
| golden-gpu | Golden Image GPU (5.2.4) |
| golden-cpu-transformers | Transformers CPU (5.2.4) |
| golden-gpu-transformers | Transformers GPU (5.2.4) |
| golden-gpu-rapids | Rapids 24.04 (5.2.4) |
| ubi-llm-rag | Red Hat LLM RAG CPU (5.2.4) |
| escu-cpu | ESCU (5.2.4) |
| spark | Spark 3.5.0 (5.2.4) |
| golden-cpu-arm | Red Hat UBI Golden CPU ARM64 (5.2.4) |
| agentic-ai | Agentic AI (5.2.4) |
When building images the build script creates an images.conf which will be placed in the <splunk>/etc/apps/mltk-container/local directory to make the image available for use in the DSDL app.
Build your own custom container images
You may extend the resources available to create your own images.
To do this add an entry to the tag_mapping.csv file which references a base image, dockerfile, requirements files and a runtime context.
tag_mapping.csv columns:
| Column | Description | Notes |
|---|---|---|
| Tag | A short name given to a combination of resources that make up an image | |
| base_image | The base image to build a DSDL container from | This may be your operating system of choice, or one that specficially supports some functionality, such as GPU libraries. This is usually gathered from a public images repository such as Dockerhub, or a private image repository. |
| dockerfile | The dockerfile to use to build this container. | Dockerfiles must be placed in the ./dockerfiles/ directory. |
| base_requirements / specific_requirements | These columns specify the names of requirements files to use from the ./requirements_files/ directory | For consistency python requirements files are spit into two. For most images base_requirements can be set to use the base_functional.txt requirements file, which contains all of the basic libraries needed for DSDL to function. specifc_requirements may then be set to install libraries appropriate for your particualr use-case or environment. For the majority of examples in the DSDL app to function you must have the libraries listed in specific_golden_cpu.txt installed. Note: the build scipt will only use these files if a compiled requirements file has not been created for that image. If you make changes to an existing requirements file please ensure you delete any compiled requirements files (./requirements_files/compiled_*) before beginning a new build. |
| runtime | This may be set to only two values: none or nvidia | nvidia only required if you are intending to use GPU functionality |
| requirements_dockerfile | This is only used if you are pre-compiling python requirements. Set this to a dockerfile with a minimal environment which will work with the proposed base image. Debian and Redhat variants are provided, see ./dockerfiles/Dockerfile.*.(redhat|debian).requirements for examples. Others may be added as needed. | |
| stanza_name | Display title used in generated image stanza output (for example, in images_conf_files/*-images.txt) | build.sh reads this as the title for the image configuration. |
Example requirements files and dockerfiles can be found in requirements_files/ and dockerfiles/ directories.
Supporting Scripts
There are a number of scripts in this repo which can help in various tasks when working with DSDL containers and collections of containers. Information about these is provided in summary in this table and in more detail below.
| Script Name | Description | Example | Notes |
|---|---|---|---|
build.sh | Build a container using a configuration tag found in tag_mapping.csv | ./build.sh golden-cpu splunk/ 5.2.4 | |
bulk_build.sh | Build all containers in a tag list | ./bulk_build.sh tag_mapping.csv splunk/ 5.2.4 | |
compile_image_python_requirements.sh | Use a base image and simplified dockerfile to pre-compute the python dependancy versions for all libraries listed in the tag's referenced requirements files | ./compile_image_python_requirements.sh golden-cpu | If the Dockerfile for the tag is not specified, the script looks for the tags Dockerfile plus the .requirements extension. If this does not exist, please create a requirements dockerfile or specifiy and appropriate requirements dockerfile. An example can be found in /dockerfiles/Dockerfile.debian.requirements |
bulk_compile.sh | Attempt to pre-compile python dependancy versions for all containers in a tag list | ./bulk_build.sh tag_mapping.csv | Makes assumptions about dockerfile names as described above. |
scan_container.sh | Scan a built container for vulnerabilities and produce a report with Trivy | ./scan_container.sh golden-cpu splunk/ 5.2.4 | Downloads the Trivy container to run the scan. |
test_container.sh | Run a set of simulated tests using Playwright on a built container. | ./test_container.sh golden-cpu splunk/ 5.2.4 | Requires the setup of a python virtual environment that can run Playwright. Specific python versions and dependancies may be required at the system level. |
Compile requirements for an image
In some cases, dependancy resolution can take a long time for an image with many python libraries. In this case you may find it faster to pre-compile python requirements. compile_image_python_requirements.sh and bulk_compile.sh scripts are provided for you to do this. Most pre-existing images will have compiled requirements shipped in this repo. If you need to rebuild an existing container with new libraries, please delete the associated compiled requirements file.
Configuraing DSDL with images.conf
After running the ./build.sh or ./bulk_build.sh scripts, <tag_name>.conf files will be created in the ./images_conf_files/ directory. If deploying a single image you may move the contents of <tag_name>.conf into /mltk-container/local/images.conf in your Splunk installation, or you may copy the contents of ./images_conf_files/images.conf into the app to configure all of the built images.
Note: for an image to be deployed it must be made available to the docker or k8s instance by publishing to a public repository (default is dockerhub) or by adding the images to an appropriate private repository.
Certificates
For development purposes the container images create self-signed certificates for HTTPS. You can replace the dltk.key and dltk.pem filed by editing the Dockerfile to build a container with your own certificates. This is one possibility to use your own certificates. There are also other options to configure your container environment with your own certificates.
Run and test your container locally
You can run your container locally, e.g. with docker run -it --rm --name mltk-container-golden-cpu -p 5000:5000 -p 8888:8888 -p 6006:6006 -v mltk-container-data:/srv splunk/mltk-container-golden-cpu:5.2.4
Further documentation and usage
Please find further information and documentation on splunkbase: Download and install the Splunk App for Data Science and Deep Learning and refer to the official documentation pages on how to install and use the app.
Contributors
Similar repos
Languages
Jupyter Notebook
83.0%
Python
14.1%
Shell
2.1%
Splunk App for Data Science and Deep Learning - container images repository
64
stars
365
commits
Jupyter Notebook
primary language
Sep 9, 2026
updated
README
Splunk App for Data Science and Deep Learning
Splunk App for Data Science and Deep Learning (DSDL) 5.2.4 formerly known as Deep Learning Toolkit for Splunk (DLTK) versions 2.3.0 - 3.9.0 and (DSDL) 5.0.0 - 5.2.4 published on splunkbase. There is also a prior DLTK version 4.x which was open sourced on GitHub but is not actively maintained. Please make use of the latest version 5.2.4
Copyright (C) 2005-2026 Splunk Inc. All rights reserved.
Author: Philipp Drieger
Contributors: Josh Cowling, Huaibo Zhao, Tatsu Murata
About
This repository contains the collection of resources, scripts, and testing frameworks that are used build and deploy the default container images used by the DSDL app. It can be used to modify, secure, update and change these containers and their build process to meet the needs of your enterprise environment.
Resources
This repository contains the container endpoint (./app), jupyter notebook configuration (./config) and examples (./notebooks), build scripts and the main Dockerfiles to create the existing pre-built container images for TensorFlow, PyTorch, NLP libraries and many other data science libraries for CPU and GPU.
Building containers
You can build your own containers with the build.sh script.
The build.sh script is invoked with at least one and up to three arguments:
./build.sh <build_configuration_tag> <repo_name> <version>
<build_configuration_tag> is used to specify the particular set of base docker image, dockerfile, base python requirements, specific python requirements and runtime setting.
These combinations can be found and added to in tag_mapping.csv.
<repo_name> allows you to specify a repo prefix which will be needed if you intend to upload images to dockerhub.
<version> allows you to specify a new version number for the completed image.
To build the default golden cpu image locally, simply run:
./build.sh golden-cpu
or specify additional arguments:
./build.sh golden-cpu local_build/ 1.0.0
For the current v5.2.4 release, the following tags are available in tag_mapping.csv. You can still add custom entries for your own builds:
build configuration tag | Description |
|---|---|
| golden-cpu | Red Hat UBI Golden CPU (5.2.4) |
| golden-gpu | Golden Image GPU (5.2.4) |
| golden-cpu-transformers | Transformers CPU (5.2.4) |
| golden-gpu-transformers | Transformers GPU (5.2.4) |
| golden-gpu-rapids | Rapids 24.04 (5.2.4) |
| ubi-llm-rag | Red Hat LLM RAG CPU (5.2.4) |
| escu-cpu | ESCU (5.2.4) |
| spark | Spark 3.5.0 (5.2.4) |
| golden-cpu-arm | Red Hat UBI Golden CPU ARM64 (5.2.4) |
| agentic-ai | Agentic AI (5.2.4) |
When building images the build script creates an images.conf which will be placed in the <splunk>/etc/apps/mltk-container/local directory to make the image available for use in the DSDL app.
Build your own custom container images
You may extend the resources available to create your own images.
To do this add an entry to the tag_mapping.csv file which references a base image, dockerfile, requirements files and a runtime context.
tag_mapping.csv columns:
| Column | Description | Notes |
|---|---|---|
| Tag | A short name given to a combination of resources that make up an image | |
| base_image | The base image to build a DSDL container from | This may be your operating system of choice, or one that specficially supports some functionality, such as GPU libraries. This is usually gathered from a public images repository such as Dockerhub, or a private image repository. |
| dockerfile | The dockerfile to use to build this container. | Dockerfiles must be placed in the ./dockerfiles/ directory. |
| base_requirements / specific_requirements | These columns specify the names of requirements files to use from the ./requirements_files/ directory | For consistency python requirements files are spit into two. For most images base_requirements can be set to use the base_functional.txt requirements file, which contains all of the basic libraries needed for DSDL to function. specifc_requirements may then be set to install libraries appropriate for your particualr use-case or environment. For the majority of examples in the DSDL app to function you must have the libraries listed in specific_golden_cpu.txt installed. Note: the build scipt will only use these files if a compiled requirements file has not been created for that image. If you make changes to an existing requirements file please ensure you delete any compiled requirements files (./requirements_files/compiled_*) before beginning a new build. |
| runtime | This may be set to only two values: none or nvidia | nvidia only required if you are intending to use GPU functionality |
| requirements_dockerfile | This is only used if you are pre-compiling python requirements. Set this to a dockerfile with a minimal environment which will work with the proposed base image. Debian and Redhat variants are provided, see ./dockerfiles/Dockerfile.*.(redhat|debian).requirements for examples. Others may be added as needed. | |
| stanza_name | Display title used in generated image stanza output (for example, in images_conf_files/*-images.txt) | build.sh reads this as the title for the image configuration. |
Example requirements files and dockerfiles can be found in requirements_files/ and dockerfiles/ directories.
Supporting Scripts
There are a number of scripts in this repo which can help in various tasks when working with DSDL containers and collections of containers. Information about these is provided in summary in this table and in more detail below.
| Script Name | Description | Example | Notes |
|---|---|---|---|
build.sh | Build a container using a configuration tag found in tag_mapping.csv | ./build.sh golden-cpu splunk/ 5.2.4 | |
bulk_build.sh | Build all containers in a tag list | ./bulk_build.sh tag_mapping.csv splunk/ 5.2.4 | |
compile_image_python_requirements.sh | Use a base image and simplified dockerfile to pre-compute the python dependancy versions for all libraries listed in the tag's referenced requirements files | ./compile_image_python_requirements.sh golden-cpu | If the Dockerfile for the tag is not specified, the script looks for the tags Dockerfile plus the .requirements extension. If this does not exist, please create a requirements dockerfile or specifiy and appropriate requirements dockerfile. An example can be found in /dockerfiles/Dockerfile.debian.requirements |
bulk_compile.sh | Attempt to pre-compile python dependancy versions for all containers in a tag list | ./bulk_build.sh tag_mapping.csv | Makes assumptions about dockerfile names as described above. |
scan_container.sh | Scan a built container for vulnerabilities and produce a report with Trivy | ./scan_container.sh golden-cpu splunk/ 5.2.4 | Downloads the Trivy container to run the scan. |
test_container.sh | Run a set of simulated tests using Playwright on a built container. | ./test_container.sh golden-cpu splunk/ 5.2.4 | Requires the setup of a python virtual environment that can run Playwright. Specific python versions and dependancies may be required at the system level. |
Compile requirements for an image
In some cases, dependancy resolution can take a long time for an image with many python libraries. In this case you may find it faster to pre-compile python requirements. compile_image_python_requirements.sh and bulk_compile.sh scripts are provided for you to do this. Most pre-existing images will have compiled requirements shipped in this repo. If you need to rebuild an existing container with new libraries, please delete the associated compiled requirements file.
Configuraing DSDL with images.conf
After running the ./build.sh or ./bulk_build.sh scripts, <tag_name>.conf files will be created in the ./images_conf_files/ directory. If deploying a single image you may move the contents of <tag_name>.conf into /mltk-container/local/images.conf in your Splunk installation, or you may copy the contents of ./images_conf_files/images.conf into the app to configure all of the built images.
Note: for an image to be deployed it must be made available to the docker or k8s instance by publishing to a public repository (default is dockerhub) or by adding the images to an appropriate private repository.
Certificates
For development purposes the container images create self-signed certificates for HTTPS. You can replace the dltk.key and dltk.pem filed by editing the Dockerfile to build a container with your own certificates. This is one possibility to use your own certificates. There are also other options to configure your container environment with your own certificates.
Run and test your container locally
You can run your container locally, e.g. with docker run -it --rm --name mltk-container-golden-cpu -p 5000:5000 -p 8888:8888 -p 6006:6006 -v mltk-container-data:/srv splunk/mltk-container-golden-cpu:5.2.4
Further documentation and usage
Please find further information and documentation on splunkbase: Download and install the Splunk App for Data Science and Deep Learning and refer to the official documentation pages on how to install and use the app.
Similar repos
Contributors
Languages
Jupyter Notebook
83.0%
Python
14.1%
Shell
2.1%