Mistral Common Release Notes

Add AudioURLChunk by @juliendenize in #120

Now you can use http(s) URLs, file paths and base64 string (without specifying format) in your content chunks thanks to AudioURLChunk !

from mistral_common.protocol.instruct.messages import AudioURL, AudioURLChunk, TextChunk, UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer

repo_id = "mistralai/Voxtral-Mini-3B-2507"
tokenizer = MistralTokenizer.from_hf_hub(repo_id)

text_chunk = TextChunk(text = "Wat do you think about this audio?")
user_msg = UserMessage(content = [AudioURLChunk(audio_url = AudioURL(url = "https://freewavesamples.com/files/Ouch-6.wav")), text_chunk])

request = ChatCompletionRequest(messages = [user_msg])
tokenized = tokenizer.encode_chat_completion(request)

# pass tokenized.tokens to your favorite audio model
print(tokenized.tokens)
print(tokenized.audios)
# print text to visually see tokens
print(tokenized.text)

Full Changelog: v1.8.0...v1.8.1

[Audio] Add audio by @patrickvonplaten in #119

Full Changelog: v1.7.0...v1.8.0

Audio chat example:

from mistral_common.protocol.instruct.messages import TextChunk, AudioChunk, UserMessage, AssistantMessage, RawAudio
from mistral_common.protocol.instruct.request import ChatCompletionRequest
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.audio import Audio
from huggingface_hub import hf_hub_download
repo_id = "mistralai/voxtral-mini"
tokenizer = MistralTokenizer.from_hf_hub(repo_id)
obama_file = hf_hub_download("patrickvonplaten/audio_samples", "obama.mp3", repo_type = "dataset")
bcn_file = hf_hub_download("patrickvonplaten/audio_samples", "bcn_weather.mp3", repo_type = "dataset")

def file_to_chunk(file: str) -> AudioChunk:
    audio = Audio.from_file(file, strict = False)
    return AudioChunk.from_audio(audio)

text_chunk = TextChunk(text = "Which speaker do you prefer between the two? Why? How are they different from each other?")
user_msg = UserMessage(content = [file_to_chunk(obama_file), file_to_chunk(bcn_file), text_chunk]).to_openai()
request = ChatCompletionRequest(messages = [user_msg])
tokenized = tokenizer.encode_chat_completion(request)
# pass tokenized.tokens to your favorite audio model
print(tokenized.tokens)
print(tokenized.audios)
# print text to visually see tokens
print(tokenized.text)

Audio transcription example:

from mistral_common.protocol.transcription.request import TranscriptionRequest
from mistral_common.protocol.instruct.messages import RawAudio
from mistral_common.audio import Audio
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from huggingface_hub import hf_hub_download
repo_id = "mistralai/voxtral-mini"
tokenizer = MistralTokenizer.from_hf_hub(repo_id)
obama_file = hf_hub_download("patrickvonplaten/audio_samples", "obama.mp3", repo_type = "dataset")

audio = Audio.from_file(obama_file, strict = False)
audio = RawAudio.from_audio(audio)
request = TranscriptionRequest(model = repo_id, audio = audio, language = "en")
tokenized = tokenizer.encode_transcription(request)
# pass tokenized.tokens to your favorite audio model
print(tokenized.tokens)
print(tokenized.audios)
# print text to visually see tokens
print(tokenized.text)

Full Changelog: v1.7.0...v1.8.0

Release notes

• Improve hf hub support by @juliendenize in #95
• Fix the Python badge by @juliendenize in #96
• [Build system] Ensure UV reads more than just py files by @patrickvonplaten in #97
• Update images.md by @juliendenize in #98
• Improve decode and deprecate to_string by @juliendenize in #99
• Fix string formatting for ConnectionError by @gaby in #101
• Fix string formatting for NotImplementedError() by @gaby in #103
• Fix error message instructions in transform_image() by @gaby in #102
• Fix spelling issues across repo by @gaby in #107
• Improve integration with HF by @juliendenize in #104
• Opening tekkenizer file with utf-8 and remove deprecation warning by @juliendenize in #110
• fix: multiprocessing pickle error with tokenizer by @NanoCode012 in #111

New Contributors

• @gaby made their first contribution in #101
• @NanoCode012 made their first contribution in #111

Full Changelog: v1.6.0...v1.6.3

Release notes

• Constantize model to tokenizer mapping (for external import) by @djsaunde in #86
• Improve the documentation by @juliendenize in #87
• Fix the documentation URL by @juliendenize in #88
• Add support for file:// protocol URIs. by @sjuxax in #85
• Add check for uv lock by @juliendenize in #89
• Add the from_hf_hub method to MistralTokenizer by @juliendenize in #90
• Add support to the openai format for Chat completions by @juliendenize in #92
• V11 Instruct by @patrickvonplaten in #91
• Use HF hub API for download by @juliendenize in #93

New Contributors

• @djsaunde made their first contribution in #86
• @sjuxax made their first contribution in #85

Full Changelog: v1.5.6...v1.6.0

Add a FastAPI app by @juliendenize in #113

We released an experimental REST API leveraging Fast API to handle requests from tokenization, through generation via calls to an engine, to detokenization.

For a detailed documentation see [https://mistralai.github.io/mistral-common/usage/experimental/].

Here is how to launch the server:

pip install mistral-common[server]

mistral_common serve mistralai/Magistral-Small-2507 \
--host 127.0.0.1 --port 8000 \
--engine-url http://127.0.0.1:8080 --engine-backend llama_cpp \
--timeout 60

Then you can see the Swagger at:
http://localhost:8000.

Full Changelog: v1.8.2...v1.8.3

Raw content

Add think chunk by @juliendenize in #122

Now you can use TextChunk and ThinkChunk in your SystemMessage or AssistantMessage :

from mistral_common.protocol.instruct.messages import SystemMessage, TextChunk, ThinkChunk
system_message = SystemMessage(content=[TextChunk(text="First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.\n\nYour thinking process must follow the template below:"), ThinkChunk(thinking="Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.", closed=True), TextChunk(text="Here, provide a self-contained response.")])

Full Changelog

Full Changelog: v1.8.1...v1.8.2