Enterprise GenAI Delivery Patterns

While Generative AI (GenAI) is only just emerging as a mature technology, we are not at the first intersection of AI and DevOps. In the drive for automation, people have used big data and machine learning (ML) to improve IT Operations processes, mostly around event correlation, anomaly detection, and causality determination (AIOps), using past inputs to predict future events. Additionally, to improve the training and delivery of machine learning models, we have seen the application of DevOps practices to make continuous delivery of ML pipelines more reliable (MLOps).

GenAI is different from traditional AI in that it is focused on generating something new instead of predicting patterns (e.g., generate a summary, generate new code, generate an answer to a question). Large language models (LLM) such as GPT-4 or Llama are focused on generating text, models like DALL-E or Stable Diffusion on generating images, and vision models such as LLAVA or GPT-vision on converting images to text. This new technology opens up a whole new set of use cases, which were hard to unlock before. During the software delivery life cycle (SDLC), almost any process can be enhanced by this new technology (i.e., using GenAI to achieve better DevOps delivery).

Instead of training specific machine learning models, these new models exhibit emergent behavior that allow them to perform a set of tasks without specifically being taught that task. This speeds up the deployment of new ideas tremendously; one can simply prototype ideas just with text. On the flip side, these models are not always focused on returning reliable answers. To compensate for this, we’ll use other techniques and we will need to put guardrails in place. To make this process more reliable, we can use existing DevOps concepts and apply them to this new technology (i.e., using DevOps to achieve better GenAI delivery).