Preface: Artificial intelligence (AI) is growing like lightning. As a I.T computer user. Maybe we enjoy the benefits of smartphone apps features empowered by AI. As a matter of fact, we do no care or without knowledge what is AI back-end operations and architecture. For example, when you buy a steamed bun at the store, you certainly don’t worry about whether there are cockroaches in the kitchen. Because you know there are public health regulations in place to prevent that. This concept also applied to AI world. So, NVIDIAs has developed a Secure Deployment Considerations Guide address to Triton Inference Server. I hope this short article has piqued your interest.

Background: AI Inference is achieved through an “inference engine” that applies logical rules to the knowledge base to evaluate and analyze new information. In the process of machine learning, there are two phases. First, is the training phase where intelligence is developed by recording, storing, and labeling information. Second, is the inference phase where the machine uses the intelligence gathered and stored in phase one to understand new data.

General-purpose web servers lack support for AI inference features.

*There is no out-of-box support to take advantage of accelerators like GPUs, or to turn on dynamic batching or multi-node inference.

*Users need to build logic to meet the demands of specific use cases, like audio/video streaming input, stateful processing, or preprocessing the input data to fit the model.

*Metrics on compute and memory utilization or inference latency are not easily accessible to monitor application performance and scale.

Triton Inference Server provides a cloud and edge inferencing solution optimized for both CPUs and GPUs. Triton supports an HTTP/REST and GRPC protocol that allows remote clients to request inferencing for any model being managed by the server.

Secure Deployment Considerations: Artificial Intelligence (AI) and Machine Learning (ML) cannot keep to yourself without the support of programming languages. Developers can deploy Triton as an http server, a grpc server, a server supporting both, or embed a Triton server into their own application. Python is one of the major code languages for AI and ML. PyTriton is a simple interface that enables Python developers to use Triton Inference Server to serve AI models, simple processing functions, or entire inference pipelines within Python code.

For Secure Deployment Considerations – Please refer to the link for details – https://github.com/triton-inference-server/pytriton

Preface: Maybe it’s a long story, but in a nutshell, this is page one. In fact, when you start studying on your first day. No matter it is an overview of AI technology. The information covers advanced mathematics, graphics and technical terminology. It will reduce your interest. In fact, the world of mathematics is complex. If a child is naturally insensitive to mathematical calculations. Could it be said that he is not suitable for working in artificial intelligence technology? The answer is not absolute. For example: Computer assembly language is difficult and complex to remember. Therefore, the solution is to develop other programming languages ​​and then convert (compile) them into machine language. This is a successful outcome in today’s technological world. Therefore, many people believe that artificial intelligence technology should help humans in other ways rather than replace human work.

Background: The machine learning process requires CPUs and GPUs. GPUs are used to train large deep learning models, while CPUs are good for data preparation, feature extraction, and small-scale models. For inference and hyperparameter tweaking, CPUs and GPUs may both be utilized.

CPU and GPU memory coherence requires data transfer, and requires defining what areas of memory are shared and with which GPUs.

Long story short: Cognition refers to the process of acquiring knowledge and understanding through thinking, experience and senses. In machine learning some neural networks will use custom non-linear activation functions or a non-standard image filter.

The technology behind facial recognition is based on deep learning, a subset of machine learning that involves training artificial neural networks to recognize patterns in data.

Ref: Non-Linear Activation Functions. The non-linear functions are known to be the most used activation functions. It makes it easy for a neural network model to adapt with a variety of data. Adaptive neural networks have the ability to overcome some significant challenges faced by artificial neural networks. The adaptability reduces the time required to train neural networks and also makes a neural model scalable as they can adapt to structure and input data at any point in time while training.

Preface: The Matrix is ​​a 1999 science fiction action film. At that time, virtual machines technology were not yet in a mature stage. IBM mainframe LPAR (Logical partitions (LPARs)) is the only implement in market successful.  Even Docker technology hasn’t even been born yet! But the film’s screenwriter seemed to predict the truth.

What is the simple definition of a matrix? The matrices are a two-dimensional set of numbers or symbols distributed in a rectangular shape in vertical and horizontal lines so that their elements are arranged in rows and columns.

Background: About five years ago, it was known that parallel computation could unlock the performance (processing speed) of supercomputers. However, programs written in traditional C language still have issue operation in this platform because C program instruction executes sequentially and do not support data parallel computation, it increases the time complexity of a program. Until Docker CUDA was born. This bottleneck appears to have been resolved. This is one of the milestones in the digital world, especially artificial intelligence technology.

Technical details: It is hard to write program in CUDA for average programmer. CUDA puts load on the programmer.

-To package GPU code in separate functions called kernel.

-Need to explicitly manage data transfer between host memory and GPU memory.

-Manual optimization of GPU memory is required.

CUDA is a parallel computing platform and programming model developed by NVIDIA for general computing on graphical processing units (GPUs). With CUDA, developers can dramatically speed up computing applications by harnessing the power of GPUs.

How does CUDA help in AI?

In addition to its components for deep learning, the CUDA Toolkit includes various libraries and components. These provide support for debugging and optimization, compiling, documentation, runtimes, signal processing, and parallel algorithms.

Official document reference: For details please refer to the link –  https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda

Preface: As of today AI tools has ChatGPT, BERT, LaMDA, GPT-3, DALL-E-2, MidJourney, and Stable Diffusion. ChatGPT was released as a freely available research preview, but due to its popularity, OpenAI now operates the service on a freemium model. It allows users on its free tier to access the GPT-3.5-based version.

Background: Legged robots, or walking machines, are designed for locomotion on rough terrain and require control of leg actuators to maintain balance, sensors to determine foot Starting from the 5G communication technology era, 5G aims to support a 100-fold increase in traffic capacity and network efficiency. So advance AI robot will rely on fast and wide coverage of Radio communication network.  Meanwhile, advanced artificial intelligence robots with decision-making and thinking mechanisms will rely on remote location big data infrastructure. So, do you think this can provide space for how humans govern this AI technology.placement and planning algorithms to determine the direction and speed of movement. Since legged robots, or walking machines installation space is limit. So this type of design  give people no so intelligence. By offloading complex computations to the cloud, robots can process vast amounts of data quickly and perform tasks that require extensive processing resources, far exceeding the capabilities of their onboard hardware.

Without 5G, there would be no real AI robots:

Coincidences are rare in science. But when we look back at the development history of 5G, we will find that this road is not smooth. On the other hand, if there is no 5G arrive in time, I believe so call artificial intelligence legged robot not easy to born. If robot cannot similar like human free to walk and without area of limitation. We cannot say our technologies is migrate to advanced digital world.

About Artificial Intelligence Endangering Human existence Value: About three years ago, when you attend seminar , the speaker will laugh when he heard AI endanger human existence value. Their comments at that time was don’t be worries too much. They are not as clever as human. The AI technology only replace the low level work job. As times goes by, the transformation of industrial process  can tell. AI technology come to our age within short period of time. On first week of Nov, 2023. The CEO of Telsa,  Elon Musk predicted that human work will become obsolete as artificial intelligence progresses, calling it “the most disruptive force in history.”

Seems we do not have choice this trend. But what we can do?

Headline new: https://www.dailymail.co.uk/sciencetech/article-12706621/When-job-taken-robot-Elon-Musk-insists-AI-mean-no-one-work-experts-reveal-careers-replaced-IMMEDIATELY-face-chop-future.html

Preface: Python has 100s of pre-built libraries to implement various Machine Learning and Deep Learning algorithms.

Background: When the word “AI” not found, Python alive everywhere in IT world. Scientist use Python programming language assists their complicated mathematics calculations. Data scientist to analyze geospatial vector data to identify a specific location. Python language is flexi and powerful. But the fundamental design weakness is well known. As a result, you should protect your AI system structure attack by threat actors. For example:
– Some Python security vulnerabilities come from Python’s open-source libraries.
– Injection: Dependency injection is a common design pattern used in most modern programming language. As a matter of fact, such weakness remains an underutilized functionality in Python.
– Command injection which can be expose through misconfig software application. As a result, the possible attack will execute arbitrary command injection, XML external entity injection and server side template injection.

About best practices for Python security

• Conducting a vulnerability scan weekly. This is the way let you identify and fix Python security vulnerability
• You should sanitize external data before use.
• Follow your Python version SDLC, DevSEC should aware of this matter.
  Additional: NVIDIA empower Artificial Intelligence competence. Staying alert with GPU vendor vulnerability management announcement.