---
title: "AI Observability with Grafana"
video_id: FbQTLxFdb6I
url: https://www.youtube.com/watch?v=FbQTLxFdb6I
workflow_page: https://workflows-of-ai.com/workflow/ai-observability-grafana
language: en
captions: generated
---

0:00 This video is sponsored by Freely Give.
0:02 If you need help with Drupal and AI,
0:04 just get in contact with us.
0:07 There's a lot of smart people and I This
0:09 is not just something I say when I say
0:11 there are smarter people around Drupal
0:13 and AI. We truly have that within the
0:15 company. So, if you need help with that,
0:17 please get in contact with us.
0:20 So, uh today I'm going to talk It's
0:22 going to be fairly short. I'm just going
0:24 to talk about
0:28 AI observability open telemetry recipe.
0:31 So, you might have seen in Drupal uh uh
0:34 in the AI module that there is AI
0:37 observability module. You might have
0:39 enabled it. You can see if you have like
0:41 a standard Drupal, it just adds it to
0:44 your normal logging.
0:46 And this is on purpose. It's using
0:48 standardized PSR logging.
0:51 What makes this powerful
0:54 is that you can actually use enterprise
0:56 solutions for this. So, in this case,
0:58 it's going to use a a recipe
1:01 to actually con- uh contact a larger
1:05 stack uh of things.
1:08 Uh and that then includes
1:11 um
1:12 It includes uh Mimir, Loki,
1:16 uh
1:16 Grafana,
1:18 Alloy open and using like open telemetry
1:21 standard. So, one thing that happens
1:23 with AI observability and the open
1:25 telemetry module is that you can
1:27 actually get
1:28 uh auto spans that follows a standard
1:31 that has been set up for AI. So, there
1:33 is also a standard how you log AI.
1:37 Uh
1:38 and to kind of understand how it works,
1:41 it's like every time Drupal makes a AI
1:43 call, the the AI observability module
1:46 then is going to capture it and send it
1:48 to open telemetry as long as you have
1:50 that module,
1:51 which lands in
1:53 Alloy. And Alloy then is a collector.
1:55 It's It's like this It's also like a um
1:59 uh
2:00 It enriches data
2:02 uh
2:02 and then can like split it three ways.
2:05 So, it creates traces
2:08 uh which is like the step-by-step story
2:11 of one request and this goes to Tempo.
2:15 It creates metrics, so like numbers over
2:18 time aggregation, request rates
2:21 uh stuff like this and goes to Mimir
2:23 uh and the actual log logs then goes to
2:26 Loki.
2:27 And what's sitting on top of all this is
2:30 Grafana.
2:31 So, that's basically where you can
2:35 visualize this and this is what I'm
2:37 going to showcase then.
2:38 Uh
2:39 I just want to say also a huge thanks to
2:43 Merge Alexey. He has been the main
2:46 driver with this. He's the main uh
2:49 maintainer of
2:51 uh
2:52 of the
2:53 um
2:54 Open Telemetry app
2:56 module, sorry, as well. Uh so, he has
3:00 been extremely good with
3:02 uh everything around the AI
3:03 observability.
3:05 Uh
3:06 he has also created an Adative plugin,
3:08 so this
3:10 uh can run then against the Adative
3:12 plugin if you want to see it end-to-end
3:14 that I have built on top of his work.
3:18 Uh but what I will also show is the
3:20 one-line installer and this will
3:22 actually show how you can go from
3:25 uh having nothing on your computer, I
3:28 mean literally nothing, a new Windows, a
3:30 new Mac uh new MacBook or a new Linux
3:33 system
3:34 and get a site up and running with this
3:37 pretty complex architecture.
3:41 You might have heard about the one-line
3:42 installer from Jamie. I will talk more
3:45 about it when it gets more stable, but
3:47 it is pretty powerful already.
3:51 So, let's get started.
3:55 So, now I'm going to show you the power.
3:57 Let me just close these tabs.
4:00 You see, there's nothing I can copy
4:02 here.
4:03 Literally,
4:04 run the one-line installer and point it
4:07 at the repo of this recipe.
4:12 And I will showcase how this works then.
4:16 So, I will just go to where my projects
4:20 are
4:22 and I will run this.
4:24 And this now
4:25 is going to download um
4:28 uh clone this repo and look for
4:31 so-called Drupal AI best practices
4:33 configs.
4:34 And see, in this case, it has two. It
4:36 has a base demo, and this sets up just
4:39 the base. You will have to set up a
4:40 provider yourself. You will have to set
4:43 up modules on how to uh utilize stuff
4:47 and so on.
4:48 Uh so,
4:50 it's not the best thing for what I want
4:52 to demo here. So, we have a demo
4:54 version. What this does, it sets up like
4:56 AI API Explorer. It sets up AI agents
5:00 with a parent and a sub agent, so we can
5:02 visualize this. It also sets up a um
5:06 uh a prompt where it starts asking you
5:08 about API keys for different providers
5:10 that you might need. So, we choose this.
5:13 And you see, it asks us right away
5:16 uh OpenAI key. So, what it's going to
5:18 do, it's going to ask for
5:20 uh different keys, and it will set up
5:22 all these providers. But, I'm just going
5:24 to set up OpenAI key.
5:26 So, you don't see it here. You can see
5:27 it's input hidden, but I have copy and
5:29 paste something. And on the other
5:30 questions, I'm just going to click
5:32 enter.
5:33 And then we're going to have uh enter a
5:35 name for the new project, and we're just
5:37 going to write
5:39 uh obs testing, so observability
5:42 testing.
5:43 And everything else
5:46 the system takes care of now. It's going
5:48 to set it up from start to finish. I can
5:52 While it's working, I can just show you
5:54 how these
5:55 um configurations looks.
5:58 So, this recipe, of course, you can
6:00 utilize this as a
6:02 any recipe
6:04 you have used before
6:06 that you just, you know, do uh um
6:09 composer require and then apply the
6:12 recipe, but you see also it has a dot
6:14 Drupal AIBP
6:16 um
6:18 uh uh directory, and this is where you
6:20 find these then, base and demo. And this
6:22 is basically the whole configuration
6:25 that sets up everything from start to
6:28 finish.
6:29 So,
6:34 I
6:35 will answer the pseudo question here,
6:38 and then
6:40 I'm going to fast forward.
7:15 And there you can see we have our site,
7:17 and this is will not look like anything
7:20 special. It's just a Drupal standard
7:22 installation.
7:24 Uh
7:25 I will log in here.
7:28 I will showcase a little bit how it has
7:33 uh it's using admin admin, so it
7:35 complains.
7:36 So, you can see, for instance, here then
7:39 if we go to keys, that it has set up
7:43 uh
7:45 Remove this annoying thing. It has set
7:48 up a environment key that is open AI API
7:51 key, so it has set it up correctly. It's
7:53 in the environment of your DDEV.
7:56 uh
7:57 In this case, it should have set up AI
8:00 API explorer as well.
8:02 So, we can start chatting here.
8:06 Hello.
8:08 Which model are you?
8:11 Can just ask something like that, right?
8:15 um
8:17 So, it's answering right, but if we go
8:19 back to terminal now, you will see
8:21 there's some information here as well.
8:23 First of all, it has jumped into OBS
8:26 testing.
8:28 It has installed Cloud Code for me, but
8:30 in a container. So, if you actually want
8:32 to
8:33 uh uh
8:34 start developing, you can actually do
8:36 this. Uh
8:37 I have chosen Cloud Code. You can choose
8:39 other
8:40 uh uh
8:41 things as well here. There's a lot of
8:43 information, but you can see there's
8:45 extra information here as well. Grafana
8:47 is here. So, we can click here.
8:50 And we get Grafana
8:52 up and running.
8:54 And here I have created some dashboard
8:56 then, and that's what we're going to
8:57 show. So, DDEV Grafana
9:00 we have agent sessions, latency
9:02 explorer, uh AI requests, and tokens and
9:07 costs.
9:08 So, all right, we did a We sent
9:10 something, so if we click here,
9:13 well,
9:14 oh, it might not actually have synced.
9:16 It might need to run uh uh brush. So,
9:19 what this does is it actually fetches
9:22 from models.dev
9:24 uh the prices for each model, so you can
9:26 actually see them here.
9:28 Uh
9:29 and if we go to AI request, you can see
9:31 we have done one request, no error rate,
9:35 average latency, average requests per
9:37 minute, and so on. We can just click
9:40 here.
9:42 And it has a refresh button. So, if you
9:44 want to see it in real time, you can do
9:46 that. I Let's see if we refresh
9:52 here.
9:57 Oh.
10:01 Oh, there we are. There is
10:03 some a little bit delayed.
10:05 So, you get some
10:07 uh data. And these are just example
10:09 data.
10:11 I was basically discussing with Fable
10:13 what makes sense to showcase. And this
10:16 is what I
10:17 uh we we came up with.
10:20 Um
10:22 You can see all the actual requests. You
10:24 can also get the raw requests, of
10:26 course, on top of that. And then there's
10:29 agent
10:30 sessions. So, here you will not see
10:32 requests because this is not a request,
10:34 right?
10:35 So,
10:37 let me see here. I think I might have to
10:40 install explorer.
10:43 Oh, yeah.
10:44 I should add this actually to the recipe
10:46 then.
10:48 Or sorry, to the demo setup.
10:50 So, we do the agent explorer.
10:53 Agents explorer, maybe.
10:57 Install it.
10:58 Uh you should use agents debugger. So,
11:00 that's probably what we should be adding
11:02 to the recipe.
11:03 And if we reload
11:06 we have the orchestrator that is the
11:09 parent agent to the fact agent. That's
11:11 just
11:12 uh something to test.
11:15 Uh
11:16 Can you ask the fact agent about a
11:20 made-up fact?
11:24 We run that, and you're going to see
11:26 here then that this is calling
11:29 um
11:30 This is going to call the
11:35 um
11:39 See.
11:40 This is going to call the sub agent.
11:43 There finally we see something.
11:45 Oh, it literally asked a made up fact.
11:49 Uh
11:51 Okay, it just comes up with some
11:53 interesting thing there. And all of a
11:56 sudden you can see how the flow is here
11:57 from parent to sub agent to parent. You
12:00 can also see
12:02 an unique ID for this session.
12:05 Uh
12:07 And then you can also see here some more
12:10 raw data, the actual queries going in
12:13 and out.
12:14 And see how they sequentially are
12:16 loaded. So, these are just some examples
12:19 on how you can set up Grafana. Of
12:21 course, if you're going to do this, you
12:23 should set it up with exactly the
12:25 measurements you want.
12:28 Um
12:30 This is just an example. Um so,
12:34 that is showcasing how you can use the
12:36 AI observability with some external
12:39 capture using standard dice
12:43 open telemetry
12:45 uh uh um structures
12:48 and also how the one-line installer
12:51 works. Thank you.