ML and data teams monitoring LLM inputs and outputs for safety issues.

LangKit is an open source text metrics toolkit that extracts signals like toxicity, jailbreaks and sentiment from prompts and responses.

Installed as a Python package and typically paired with the whylogs data logging library or the WhyLabs platform.

Released by WhyLabs and still actively maintained with around 1,000 GitHub stars.

Turns unstructured LLM text into structured metrics so teams can detect prompt injection and quality drift.

Depends on whylogs for profiling and integrates with the commercial WhyLabs monitoring platform for dashboards and alerts.

Apache 2.0 open source library that runs locally, sending profiles to the WhyLabs SaaS platform is optional and requires an account.