bomonike

Even your AI prompts can be done better with AI with Declarative Self-improving Python (DPSy).

Overview

• Strategic framework
• Tutorials
• Promptmetheus
• What is DSPy?
• References
• How DSPy Works
• Prompt Workflow
• Signatures
• Metric
• Cohare’s ColBERT
• References

Do you think that you’ve “AI proofed” your job because you got results from prompt text?

Wrong. LLMs can outperform humans at prompt optimization as well. Even writing prompts can be optimized (done better and faster) by AI.

“Don’t bother figuring out what special magic combination of words will give you the best performance for your task. Just develop a scoring metric, then let the model optimize itself.” – Rick Battle, VMWware “Don’t Start a Career as an AI Prompt Engineer.” IEEE Spectrum May 2024.

The object of this article is so you can add to your LinkedIn profile / resume something like this:

Achieved higher-rated results 3X quicker by leading optimization of LLM prompts through automated evaluation, using DSPy.

Strategic framework

VIDEO: Google Prompting Essentials 9-hour video Coursera by Googlers. present their memonic2:

Stage	Mememonic1	Mememonic2
Task	Tiny	Thoughtfully
Context	Crabs	Create
References	Ride	Really
Evaluate	Enormous	Excellent
Iterate	Iguanas	Inputs

The first memonic is from Tina Huang’s “Google’s 9 Hour AI Prompt Engineering Course In 20 Minutes”
https://f.io/0x4JnYKV

Tutorials

Tutorials FREE on Infosys Springboard Digital Academy: 7h28m “Prompt Engineering” This text-based course intends to teach learners the best practices of Prompt Design to extract more relevant and useful output from Large Language Models such as OpenAI GPT-3 /chatGPT, etc.

Promptmetheus

What is DSPy?

DSPy is an acronym (pronounced “Dee Ess Py”), standing for Declarative Self-improving Python. It is a utility library open sourced in Dec 2022 by Standford’s NLP group in the Stanford Artificial Intelligence Laboratory (SAIL):

https://github.com/stanfordnlp/dspy

As of this writing, it had 261 contributors, led by @lateinteraction members Omar Khatteb and with Michael Ryan and Josh Patel.

BTW, the name is also a music troupe. So search for “dspy ai”.

References

According to these sources:

1. https://dspy.ai
2. https://arvix.org/pdf/2310.03714 DSPy Self-improving Pipelines
3. https://discord.gg/XCGy2WDCQB server invitation
4. Underlying research at https://github.com/stanfordnlp/dspy?tab=readme-ov-file#-citation–reading-more
5. “Prompt Engineering is Dead; Build LLM Applications with DSPy Framework” by Matt Yates on the Databricks channel
6. this
7. https://www.youtube.com/watch?v=QdA-CRr_oXo Learn by Building AI “A Gentle Introduction” Part 1
8. https://www.youtube.com/watch?v=NoaDWKHdkHg&t=9m6s by MLOps.community interview Omar Khattab
9. https://www.youtube.com/watch?v=_ROckQHGHsU by Neural Breakdown with AVB
10. https://www.youtube.com/watch?v=s85YY3myQLw by FunctionalTV
11. https://arvix.org/pdf/2309.03409 LLMs as optimizers
12. https://arvix.org/pdf/2405.06683 ERAGent
13. https://www.youtube.com/watch?v=Y94tw4eDHW0 by Cohere (Canada) ColBERT search paradigm
14. https://www.youtube.com/watch?v=Jfpxjg8xj9w by Fahd Mirza “Easiest Tutorial”
15. https://www.youtube.com/watch?v=ayHGacm5R_w by The AI Advantage “You probably don’t need prompt engineering”
16. https://www.youtube.com/watch?v=zBb8XQZ4ksI by Kamalraj MM
17. https://www.youtube.com/watch?v=s85YY3myQLw by FunctionalTV
18. https://www.youtube.com/playlist?list=PLYHC03VXva9sCKIAcozNMrtiKwjl6EQR- June 2024
19. https://www.youtube.com/watch?v=7EptCzJ9AL8 by Hacking AI
20. https://www.youtube.com/watch?v=wds11xm4ruo by Data Science in Everyday Life

21. https://www.youtube.com/watch?v=6rN9ozzdT3A “DSPy: MOST Advanced AI RAG Framework with Auto Reasoning and Prompting” by Mervin Praison on python3 -m pip install dspy dspy-ai openai rich cloudinary, hotpot

    by Discover AI:

22. https://www.youtube.com/watch?v=U5ZuMZkZBSY Understand DSPy: Programming AI Pipelines

23. https://www.youtube.com/watch?v=6YtdtjQD1r0 DSPy: Advanced Prompt Engineering?

    by Greg Loughnane on AI Makerspace

24. https://www.youtube.com/watch?v=W_fscw3PXjg DSPy: Advanced RAG?

“You wouldn’t hand-select weights of your neural network, so don’t hand-select your prompts.” DSPy’s job is to expand signatures into prompts. DSPy also parses typed outputs.

How DSPy Works

DSPy is basically an optimising compiler that reads declarative code (signatures) to generate AI prompt text based on its predefined templates.

DSPy shifts the focus of work from tinkering with brittle prompt strings and messy incidental choices to programming with structured and declarative natural-language modules.

• “Self-improving” means that Python framework for iteratively improving prompts using Python programming.

• “Declarative” means for every AI component in your system, you specify input/output behavior as a signature and select a module to assign a strategy for invoking your LM.

DSPy is based on PyTorch.

Each template consists of:

Given the Context and Question, respond with Query.

DSPy uses optimizers VIDEO:

• dspy.BootstrapFewShotWithRandomSearch
• dspy.COPRO and dspy.MIPROv2(Multi-Instruction PRoposer Optimiser version 2) induction of revised instructions VIDEO
• dspy.BootstramFinetune for multi-stage fine tuning

Compose different modules together into ergonomic, portable, and optimizable AI systems.

Don’t just write a prompt and call it good enough.

Improve your outputs by improving the depth and breath of your testing processes.

There are several levels of testing:

Within enterprises, it is now commonplace for automation to run the various tests:

Prompt Workflow

https://www.lycee.ai/blog/getting-started-with-dspy

“Run AI models locally without an expensive GPU” by Zen van Riel

Signatures

Dpsy asks that you write your system in normal Python, but express AI steps in the form of DSPy signatures.

As with other languaes, a signature defineds the input and output parameters of a function.

DSPy handles:

(i) applying prompting strategies like CoT or ReAct over your signatures,

(ii) expanding your signatures into prompts for your LM, and

(iii) parsing the LM outputs, enforcing structure/types to give you back the values you asked for.

Metric

What’s unique with DSPy is that it asks for a metric that evaluates your system’s outputs.

The DSPy Optimizer has you pick will

(i) build & explore better instructions for every module,

(ii) generate good examples for every prompt, and/or

(iii) finetune the LM weights.

Cohare’s ColBERT

Cohare https://llm.university

Programming Foundation Models with DSPy / Multivector Semantic Search with ColBERT - Omar Khattab at NEURIPS 2024

ColBERT search paradigm is based on paper: https://arxiv.org/pdf/1905.10650

ColBERT provides evaluation? Yes, see https://colbert.rocks

https://www.youtube.com/watch?v=Dt3H2ninoeY Omar Khattab, DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines by FunctionalTV

DSPy and ColBERT with Omar Khattab! - Weaviate Podcast #85 by Weaviate • Vector Database

References

1. https://www.youtube.com/watch?v=Zv4LjO8teqE&t=856s&pp=ygUKZHBzeS5weSBhaQ%3D%3D
2. Lindy

Extends RAG (Retrieval-Augmented Generation))

1. https://www.youtube.com/watch?v=AxfmzLz9xXM “An AI Prompt Engineer Shares Her Secrets” by Fortune Magazine