Preface: The nvJPEG library provides low-latency decoding, encoding, and transcoding for common JPEG formats used in computer vision applications such as image classification, object detection and image segmentation. Background: The nvJPEG library enables the following functions: use the JPEG image data stream as input; retrieve the width and height of the image from the data…

Preface: Standardized AI/ML model packaging: With OCI artifacts, models can be versioned, distributed, and tracked like container images. This promotes consistency and traceability across environments.Docker Desktop, specifically through its Docker Model Runner feature, can be used to run various AI models, particularly Large Language Models (LLMs) and other AI models that can be packaged as…

Preface: LLMs are built on machine learning: specifically, a type of neural network called a transformer model. How do LLMs read PDFs? The first step was to extract the text blocks from the PDF using pdfplumber . Each text block came with its coordinates, which allowed to analyze their spatial relationships. Next, I created a “window” around each text block…

Preface: For years, OpenAI’s GPT series has been a dominant force, while NVIDIA’s Megatron-LM has provided a powerful framework for training these massive models. NVIDIA Megatron-LM faces competition from several other frameworks especially Microsoft DeepSpeed, Hugging Face Accelerate, JAX/Flax and PyTorch Lightning. Both PyTorch Lightning and NVIDIA Megatron-LM are built on top of the PyTorch…