Skip to content

Commit

Permalink
Merge branch 'main' into fix-json-dump-unicode
Browse files Browse the repository at this point in the history
  • Loading branch information
@logan-markewich
logan-markewich committed Jun 19, 2024
2 parents 39c960f + d8e7c90 commit 23fe3f9
Show file tree
Hide file tree
Showing 26 changed files with 1,034 additions and 240 deletions.
133 changes: 67 additions & 66 deletions docs/docs/examples/llm/ai21.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,7 @@
"source": [
"# AI21\n",
"\n",
"This notebook shows how to use AI21's newest model - Jamba-Instruct. Along with support for the previous model family - Jurassic (J2-Mid and J2-Ultra).\n",
"\n",
"By default - jamba-instruct model is used. If you want to use the Jurassic models, you can specify the model name as \"j2-mid\" or \"j2-ultra\"."
"This notebook shows how to use AI21's foundation models in LlamaIndex. The default model is `jamba-instruct`. If you want to use the older Jurassic models, specify the model name \"j2-mid\" or \"j2-ultra\"."
]
},
{
Expand All @@ -30,7 +28,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.\n"
"If you're opening this Notebook on colab, you probably need to install LlamaIndex 🦙.\n"
]
},
{
Expand All @@ -55,40 +53,32 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setting API Key"
"## Setting the AI21 API Key"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When creating the AI21 instance, you can pass the API key as a parameter or by default, it will be consumed from the environment variable `AI21_API_KEY`."
"When creating an `AI21` instance, you can pass the API key as a parameter. If not provided as a parameter, it defaults to the value of the environment variable `AI21_API_KEY`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Paul Graham is a computer scientist, entrepreneur, and author. He is best known as the co-founder of Y Combinator, a venture capital firm that provides seed funding to early-stage startups. He is also a well-known essayist and writer on various topics such as programming, startups, and technology. He has written several influential essays on these topics, which have been widely read and discussed in the tech community.\n",
"Paul Graham is a computer scientist, entrepreneur, and essayist known for his work in the field of computer programming and his contributions to the startup ecosystem. He is the co-founder of Viaweb, which was later sold to Yahoo! and became Yahoo! Store. Graham is also recognized for his essays on various topics related to technology, startups, and the nature of innovation, which are widely read and discussed in the tech community. He co-founded Y Combinator, one of the most prominent startup accelerators in the world, which has helped launch over a thousand startups, including notable companies like Airbnb, Dropbox, and Reddit. His insights and writings on entrepreneurship and technology have made him a respected and influential voice in the tech world.\n"
]
}
],
"outputs": [],
"source": [
"import os\n",
"from llama_index.llms.ai21 import AI21\n",
"\n",
"api_key = \"Your-api-key\"\n",
"# EITHER\n",
"api_key = <YOUR API KEY>\n",
"os.environ[\"AI21_API_KEY\"] = api_key\n",
"\n",
"llm = AI21()\n",
"\n",
"# Or\n",
"# OR\n",
"llm = AI21(api_key=api_key)"
]
},
Expand All @@ -99,6 +89,13 @@
"#### Call `chat` with a list of messages"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Messages must be listed from oldest to newest, starting with a `user` role message and alternating between `user` and `assistant` messages."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -113,9 +110,10 @@
" ChatMessage(\n",
" role=\"assistant\", content=\"Arrrr, matey! How can I help ye today?\"\n",
" ),\n",
" ChatMessage(role=\"user\", content=\"What is your name\"),\n",
" ChatMessage(role=\"user\", content=\"What is your name?\"),\n",
"]\n",
"\n",
"# Use `preamble_override` to specify the voice and tone of the assistant.\n",
"resp = AI21(api_key=api_key).chat(\n",
" messages, preamble_override=\"You are a pirate with a colorful personality\"\n",
")"
Expand All @@ -125,7 +123,15 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"assistant: Arrrr, ye can call me Captain Jamba! I be a friendly pirate AI, here to help ye with any questions ye may have.\n"
]
}
],
"source": [
"print(resp)"
]
Expand Down Expand Up @@ -158,8 +164,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"an American computer scientist, essayist, and venture capitalist. He is best known for his work on Lisp, programming language design, and entrepreneurship. Graham has written several books on these topics, including \" ANSI Common Lisp\" and \" Hackers and Painters.\" He is also the co-founder of Y Combinator, a venture capital firm that invests in early-stage technology companies.\n"
"Paul Graham is a computer scientist, entrepreneur, and writer. He is best known as the co-founder of Y Combinator, a venture capital firm that has funded over 2,000 startups, including Dropbox, Airbnb, and Reddit. Graham is also known for his essays on technology, startups, and programming languages, which he publishes on his website paulgraham.com. He is a strong advocate for the use of technology to improve people's lives and has written extensively about the importance of entrepreneurship and innovation.\n"
]
}
],
Expand All @@ -171,14 +176,41 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configure Model"
"## Call Async Methods"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core.llms import ChatMessage\n",
"from llama_index.llms.ai21 import AI21\n",
"\n",
"prompt = \"What is the meaning of life?\"\n",
"\n",
"messages = [\n",
" ChatMessage(role=\"user\", content=prompt),\n",
"]\n",
"\n",
"chat_resp = await AI21(api_key=api_key).achat(messages)\n",
"\n",
"complete_resp = await AI21(api_key=api_key).acomplete(prompt)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Adjust the model behavior"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Configure the parameter passed to the model like - `max_tokens`, `temperature`"
"Configure parameters passed to the model to adjust its behavior. For instance, setting a lower `temperature` will cause less variation between calls. Setting `temperature=0` will generate the same answer to the same question every time. "
]
},
{
Expand Down Expand Up @@ -212,41 +244,14 @@
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"an American computer scientist, essayist, and venture capitalist. He is best known for his work on Lisp, programming language design, and entrepreneurship. Graham has written several books on these topics, including \" ANSI Common Lisp\" and \" Hackers and Painters.\" He is also the co-founder of Y Combinator, a venture capital firm that invests in early-stage technology companies.\n"
"Paul Graham is an American computer scientist, entrepreneur, and author. He is best known for his work in the field of computer programming languages, particularly the development of the Arc programming language. He is also a co-founder of the influential startup accelerator Y Combinator, which has helped launch many successful technology startups.\n"
]
}
],
"source": [
"print(resp)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set API Key at a per-instance level\n",
"If desired, you can have separate LLM instances use separate API keys."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from llama_index.llms.ai21 import AI21\n",
"\n",
"llm_good = AI21(api_key=api_key)\n",
"llm_bad = AI21(model=\"j2-mid\", api_key=\"BAD_KEY\")\n",
"\n",
"resp = llm_good.complete(\"Paul Graham is \")\n",
"print(resp)\n",
"\n",
"resp = llm_bad.complete(\"Paul Graham is \")\n",
"print(resp)"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -258,7 +263,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"using `stream_chat` method"
"Stream generated responses at one token per message using the `stream_chat` method."
]
},
{
Expand Down Expand Up @@ -289,25 +294,21 @@
"name": "stdout",
"output_type": "stream",
"text": [
"None Once upon a time, there was a pirate named Captain Jack. He was a very colorful pirate with a love for adventure and treasure. He sailed the seven seas, always searching for his next big score.\n",
"\n",
"One day, he stumbled upon a map that led to a hidden island filled with gold and riches beyond his wildest dreams. He gathered his loyal crew and set sail towards the island.\n",
"\n",
"As they approached the island, they were met with treacherous waters and dangerous obstacles. But Captain Jack was determined to find the treasure, and he wouldn't let anything stand in his way.\n",
"None Once upon a time, in a faraway land, there was a brave and adventurous pirate named Captain Jack. He had a colorful personality and was known for his quick wit and cunning.\n",
"\n",
"Finally, after days of searching, they found the hidden island. They docked their ship and made their way through the dense jungle, following the map to the treasure.\n",
"One day, Captain Jack set sail on his trusty ship, the Black Pearl, in search of treasure. He and his crew sailed across treacherous waters and battled fierce storms, but they never gave up.\n",
"\n",
"As they approached the treasure, they were met with a group of fierce warriors who guarded the treasure. But Captain Jack and his crew were not afraid. They fought bravely and defeated the warriors, finally reaching the treasure.\n",
"After many long days at sea, they finally found the island where the treasure was said to be buried. They anchored their ship and set out on foot, armed with their trusty swords and pistols.\n",
"\n",
"They opened the treasure chest to find it filled with gold, jewels, and other valuable treasures. They were overjoyed and celebrated their victory.\n",
"As they made their way through the dense jungle, they encountered all manner of dangerous creatures, from venomous snakes to giant spiders. But Captain Jack and his crew were not afraid. They fought their way through, determined to reach the treasure.\n",
"\n",
"But Captain Jack knew that they couldn't stay on the island forever. They loaded their ship with the treasure and set sail back home.\n",
"Finally, after what seemed like an eternity, they arrived at the spot where the treasure was supposed to be buried. They dug deep into the earth, their hearts pounding with excitement. And at last, they struck gold!\n",
"\n",
"As they sailed away, they were met with a fierce storm that threatened to sink their ship. But Captain Jack was determined to make it back home with the treasure. He fought against the storm with all his might, and eventually, they made it through.\n",
"They had found the treasure! Captain Jack and his crew were overjoyed. They gathered up as much gold and jewels as they could carry and set sail for home.\n",
"\n",
"Captain Jack and his crew finally made it back to their home port, where they were greeted as heroes. They had returned with a treasure that would make them rich beyond their wildest dreams.\n",
"As they sailed back to their home port, Captain Jack regaled his crew with stories of their adventures and the dangers they had overcome. They laughed and sang and drank to their good fortune.\n",
"\n",
"And so, Captain Jack and his crew lived happily ever after, sailing the seas and searching for their next big adventure."
"When they finally arrived back home, Captain Jack and his crew were hailed as heroes. They had risked everything to find the treasure and had returned victorious. And Captain Jack, with his colorful personality, was the most celebrated of all."
]
}
],
Expand All @@ -327,7 +328,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The type of the tokenizer is determined by the name of the model "
"Different models use different tokenizers."
]
},
{
Expand Down
93 changes: 92 additions & 1 deletion docs/docs/examples/pipeline/query_pipeline.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@
"- Chain together prompt and LLM\n",
"- Chain together query rewriting (prompt + LLM) with retrieval\n",
"- Chain together a full RAG query pipeline (query rewriting, retrieval, reranking, response synthesis)\n",
"- Setting up a custom query component"
"- Setting up a custom query component\n",
"- Executing a pipeline step-by-step"
]
},
{
Expand Down Expand Up @@ -1225,6 +1226,96 @@
"source": [
"print(str(output))"
]
},
{
"cell_type": "markdown",
"id": "5a1186f8",
"metadata": {},
"source": [
"## Stepwise Execution of a Pipeline\n",
"\n",
"Executing a pipeline one step at a time is a great idea if you:\n",
"- want to better debug the order of execution\n",
"- log data in between each step\n",
"- give feedback to a user as to what is being processed\n",
"- and more!\n",
"\n",
"To execute a pipeline, you must create a `run_state`, and then loop through the exection. A basic example is below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fec7873b",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core.query_pipeline import QueryPipeline\n",
"from llama_index.core import PromptTemplate\n",
"from llama_index.llms.openai import OpenAI\n",
"\n",
"# try chaining basic prompts\n",
"prompt_str = \"Please generate related movies to {movie_name}\"\n",
"prompt_tmpl = PromptTemplate(prompt_str)\n",
"llm = OpenAI(model=\"gpt-3.5-turbo\")\n",
"\n",
"p = QueryPipeline(chain=[prompt_tmpl, llm], verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "900210e6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1. Infernal Affairs (2002) - The original Hong Kong film that inspired The Departed\n",
"2. The Town (2010) - A crime thriller directed by Ben Affleck\n",
"3. Mystic River (2003) - A crime drama directed by Clint Eastwood\n",
"4. Goodfellas (1990) - A classic mobster film directed by Martin Scorsese\n",
"5. The Irishman (2019) - Another crime drama directed by Martin Scorsese, starring Robert De Niro and Al Pacino\n",
"6. The Departed (2006) - The Departed is a 2006 American crime film directed by Martin Scorsese and written by William Monahan. It is a remake of the 2002 Hong Kong film Infernal Affairs. The film stars Leonardo DiCaprio, Matt Damon, Jack Nicholson, and Mark Wahlberg, with Martin Sheen, Ray Winstone, Vera Farmiga, and Alec Baldwin in supporting roles.\n"
]
}
],
"source": [
"run_state = p.get_run_state(movie_name=\"The Departed\")\n",
"\n",
"next_module_keys = p.get_next_module_keys(run_state)\n",
"\n",
"while True:\n",
" for module_key in next_module_keys:\n",
" # get the module and input\n",
" module = run_state.module_dict[module_key]\n",
" module_input = run_state.all_module_inputs[module_key]\n",
"\n",
" # run the module\n",
" output_dict = module.run_component(**module_input)\n",
"\n",
" # process the output\n",
" p.process_component_output(\n",
" output_dict,\n",
" module_key,\n",
" run_state,\n",
" )\n",
"\n",
" # get the next module keys\n",
" next_module_keys = p.get_next_module_keys(\n",
" run_state,\n",
" )\n",
"\n",
" # if no more modules to run, break\n",
" if not next_module_keys:\n",
" run_state.result_outputs[module_key] = output_dict\n",
" break\n",
"\n",
"# the final result is at `module_key`\n",
"# it is a dict of 'output' -> ChatResponse object in this case\n",
"print(run_state.result_outputs[module_key][\"output\"].message.content)"
]
}
],
"metadata": {
Expand Down
2 changes: 1 addition & 1 deletion llama-index-core/llama_index/core/async_utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def asyncio_run(coro: Coroutine) -> Any:
If there is no existing event loop, creates a new one.
"""
try:
loop = asyncio.get_running_loop()
loop = asyncio.get_event_loop()
if loop.is_running():
raise RuntimeError(
"Nested async detected. "
Expand Down
Loading

0 comments on commit 23fe3f9

Please sign in to comment.