Function calling

Gray Swan's Completions API supports function calling (a.k.a tool usage).

Function calling example

First, you must specify the available functions using JSON schemas. Here is an example:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_temperature",
            "description": "Get the temperature for a given date and place",
            "parameters": {
                "date": {
                    "type": "string",
                    "description": "Date in ISO 8601 format"
                },
                "location": {
                    "type": "string",
                    "description": "Location"
                },
                "units": {
                    "type": "string",
                    "description": "Units to use for temperature"
                },
            },
            "required": ["date"]
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_moon_phase",
            "description": "Get the moon phase on a given date",
            "parameters": {
                "date": {
                    "type": "string",
                    "description": "Date in ISO 8601 format"
                }
            },
            "required": ["date"]
        }
    }
]

When using the Completions client, you can provide the list of tools as an argument:

import os
from gray_swan import GraySwan
GRAYSWAN_API_KEY = os.environ.get("GRAYSWAN_API_KEY")
 
client = GraySwan(
    api_key=GRAYSWAN_API_KEY,
)
completion_create_response = client.chat.completion.create(
    messages=[{"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "What is the weather like in Ohio on April 25th, 2024?"}
              ],
    model="cygnet",
    tools=tools
)

In the model's response, a message might contain some number of function calls. These can be accessed as follows:

function_call = completion_create_response.choices[0].message.tool_calls[0].function
 
function_name = function_call.name
function_args = function_call.arguments

As an example, the function call returned by the model might have name get_temperature and arguments {"date": "2024-04-25", "location": "Ohio", "units": "fahrenheit"}.

After receiving the function call from the model, if this corresponds to a tool you have access to, you can pass the arguments from the tool and get a result. Then, if you wish to pass the result back to the model, you can use the tool role.

For instance, if you stored the return result in a variable temperature_result:

next_response = client.chat.completion.create(
    messages=[{"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "What is the weather like in Ohio on April 25th, 2024?"},
              {"role": "tool", "name": "get_temperature", "content": temperature_result}
              ],
    model="cygnet",
    tools=tools
)

Streaming with function calling

Using the same tools as above, here is an example of streaming with function calling. For simplicity, we assume there is only one function call.

import os
from gray_swan import GraySwan
GRAYSWAN_API_KEY = os.environ.get("GRAYSWAN_API_KEY")
 
client = GraySwan(
    api_key=GRAYSWAN_API_KEY,
)
response = client.chat.completion.create(
    messages=[{"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "What is the weather like in Ohio on April 25th, 2024?"}
              ],
    model="cygnet",
    tools=tools,
    stream=True
)
 
function_name = ""
function_args = ""
for r in completion_create_response:
    if r.choices[0].delta.tool_calls:
        function_name += r.choices[0].delta.tool_calls[0].function.name
        function_args += r.choices[0].delta.tool_calls[0].function.arguments

When streaming, the name will always be provided before the arguments. If there are multiple function calls, you can detect the start of the next function call when the previous arguments has no unclosed curly braces.

Function calling

Function calling example

Streaming with function calling

On this page