The Monkey Patching Podcast: Going Bananas on AI, Data, LLMs & Tech

Murilo (00:08)
Hi everyone, welcome to the Monkey Patching Podcast where we go bananas about all things lazy inputs, physical AI and more. My name is Marilo, I'm joined by my friend Bart, hey Bart.

Bart (00:18)
Hey, Merdan.

I'm doing well, I'm doing well. What about you?

Murilo (00:23)
How are you?

I'm feeling better, I'm recovered almost from the sickness, indeed stronger than ever so happy to be here.

Bart (00:28)
It's It's good.

you're a bit on potato quality webcam.

Murilo (00:37)
Yeah,

maybe it's, I don't know if we talked about it on the monkey patching, but usually we have the iPhone, right, to connect. I even have like a device here as a webcam. Continuity camera. Yeah. And it's like as the Apple ecosystem, right? It works super good. It just works except when it doesn't. And then when it doesn't, it's like, it just doesn't, like you don't know how to debug it because usually it just connects, right?

Bart (00:47)
as a webcam, right? The continuity camera, it's called.

But it's really a problem with Chrome, right? It's not really Apple, like it's Chrome.

Murilo (01:08)
Yeah, is

just a Chrome thing. So I even switched to Safari because the Apple features actually work better as well with Safari, like the passwords and everything. But like with Chrome is always a hustle. Like you have to close all the apps, put on horizontal, and then sometimes it works, sometimes it doesn't. There's no support as well. think we only saw one article, I don't even know if it was a Reddit thing, that someone just kind of said.

Bart (01:33)
That's weird that it's not

something that is widely discussed, huh?

Murilo (01:36)
Exactly, it's like in Chrome as well like it's like the biggest browser and like I don't know so it's weird so we did try

Bart (01:39)
⁓ So we tried ⁓ restarting for

five minutes and then we said this is maybe just good for like having a like a retro 90s Right? It looks authentic.

Murilo (01:45)
slices it.

Yeah, it's like never look better, you know, you can you don't see my pores today, so it's

All right, what do we have Bart should you start should I start?

Bart (01:58)
I'll kick it off X-Peng.

Not sure if I'm pronouncing that correctly actually. Do you know? X-bang. Sorry?

Murilo (02:02)
I don't know either. I don't know. Xpink sounds right.

Xpink sounds right, yeah.

Bart (02:08)
XBank used its 2025 AI day to pitch a pivot to physical AI, unveiling VLA 2.0, a robot taxi program, the next-gen iron humanoid and a flying car, signaling ambitions beyond EVs. The company touts concrete mass production timelines and even named Volkswagen the launch customer for VLA 2.0. ⁓

Murilo (02:31)
Cool,

Bart (02:35)
If you follow the news a little bit, think what people have probably followed is the unveiling of the humanoid robot, ⁓ which moves very, very human-like.

Murilo (02:44)
Yes.

Yes.

Bart (02:51)
Because it moves very human-like, they also wanted to, well it was a bit of a marketing of course, but they wanted to prove that it was not actually a human in a suit. They cut open basically the clothing on the leg of the robot while on stage.

Murilo (03:05)
Well, I saw that video, that was LinkedIn, or sorry, LinkedIn, X post, right? There's also a promotional video and it even shows like the thing dancing a bit, right? And I think I saw afterwards it says, this is not sped up. This is really the sophisticated robot really, right? So it looks really cool. Yeah.

Bart (03:27)
Yeah, it looks very impressive, ⁓

So what they want to do with it is very near term is that they are thinking for next year for commercial service rules. So like quote unquote simple things like store greet, guided tours, shopping assistance, maybe like a traffic diversion for like when there are like big football games or something like that. And there's also gonna be industrial pilots at BOA Steel, a steel manufacturer.

for inspection work.

Murilo (03:59)
So they will use the robot to perform inspections. That's how.

Bart (04:02)
That's how I understand it, yeah. Exactly.

It's for now still quite, let's say, small batches, but they want to do mass production by end of 2026. No official price yet announced, but there are some rumors that it's gonna be around 150k per unit.

Murilo (04:17)
Are you gonna buy one?

Bart (04:17)
bit steeper to like greet me at my door. Welcome home, Bart. Ching.

Murilo (04:18)
Exactly.

Like shh, shh, shh, know, just waving. They also had like flying cars, which is basically like a helicopter, right? It's more like a helicopter than a car. They also had like self-driving cars, I think I saw on the promotional video thing. Right? So they also, and I think this is all self- flying, self-driving, right? As well.

Bart (04:29)
Hmm.

Yeah, exactly. So they not sure how integrated all this is, but they presented very integrated, so they have what they call their robotics techs, which is like...

VLT, is a vision language task model. have VLA, vision language action model, and a VLM, vision language model. And that stack they reuse for the humanoid, for autonomous driving. But like I said, I'm not sure how actually reusable that is, if it's just presented that way, that they have like this AI brain that they can use for every circumstance.

Murilo (05:13)
Yeah.

In any case, it looks

cool.

Bart (05:21)
It's a company that in the past has focused already a lot on autonomous driving assistance. And I think because of that also has a lot of access to training data. I had some information on how much, which was actually quite interesting. And they have around, they sort of trained around on 100 million real driving clips.

Murilo (05:28)
Okay.

Bart (05:44)
which is significant, right? The means that you already have, like, sorry? That's how I understand it, that's vision-based, yeah.

Murilo (05:46)
Yeah, indeed. And it's all vision-based. It's all vision-based then.

Did you see the video of the Tesla? They had a basically big panel with painted like the coyote. What's the name of those joints? White coyote, you know how there was a scene that they paint a tunnel in a rock and then it just hits and then they tried the same thing with the Tesla and actually the Tesla didn't stop at all. just went through the... was like a bit... Yeah, indeed. So they were just trying to say like, look how... Because it's vision based indeed. But I mean...

Bart (06:02)
Wiley Gioeddi.

Mm-hmm.

wow, seriously?

Because it's vision based, right? And not sensor

based.

Murilo (06:23)
Because, yeah, I don't know. think they're trying to make a point, right? That Teslas are unreliable. I think it was also when Elon was taking a decline, right? ⁓ But I still feel like it's probably more reliable than Sonar, right? I mean, I realistically probably need both to really be, to cover your bases, right? But I'm not sure if, I don't know. Because I know there was a project that touched a bit on these Sonars, right?

Bart (06:32)
Hmm.

Murilo (06:47)
And it was super hard to debug because the signal's very noisy. And then sometimes you get like a signal and you don't know if it's, you cannot diagnose it very well, right? And I think if you have images, it's much easier to build something that is more robust that you can explain why this and why that is happening.

Bart (07:02)
Let's see. I mean, after feeling like that fully autonomous driving, it gets promised every year and we still haven't seen it.

Murilo (07:02)
So, let's see.

Yeah, that's true. That's true.

I was actually thinking of that when I was listening, watching the video as well. It's like, like it will happen one point, right? Like if you have all the time with infinity in the future.

Bart (07:19)
But if

you look at what is already happening with Waymo in the US, these are autonomous caps. Let's see what it gets here.

Murilo (07:24)
This show.

That's true, that's I think,

yeah, exactly. I think that's the thing, right? I think I'm just surprised that it's taking too long. And one last question I wanted to ask as well is robots with humanoids. Sometimes I wonder like, we're just doing this so they're human friendly, right? But as a mechanical thing, is this the best way to design robots?

Bart (07:49)
I think this is very good question. Take for example the example that they had there, support or whatever kind of support in a steel factory.

Murilo (07:51)
You know what saying? Like, we're...

Bart (08:01)
I think it's probably going to be more cost efficient to build something that is specific for a certain task and it's probably not going to look like a humanoid. ⁓

Murilo (08:09)
Exactly, right? Like

has, I mean, just the fact like put wheels and then you don't have to deal with the whole way of people walking, right? Like it's just, I don't know.

Bart (08:15)
Yeah.

At the same point, long long term, but it's probably not in the coming years, humanoid form is maybe way more flexible. You purchase one unit for was it, 150k, and then you can do all kinds of things with it instead of having to buy something that is dedicated for a specific task.

Murilo (08:37)
Well, I mean, yeah, I mean, it can do all sorts. It can do all sorts of things that you can imagine yourself doing, right? So it's really a replacement to you. But there are other things as well that you cannot do, right? And maybe you want something that is better suited for that task, like fitting to tight spaces or I don't know.

Bart (08:39)
But that means long term.

Yeah, I want a

robot with a jetpack on its back so it can also fly.

Murilo (08:57)
That's it.

That's it. Okay. No more questions. That's it. So it needs to be like huge shoulders, tiny neck and you can just like... All right. Cool. Yeah. Nice to see this. I mean, it's fun to see these things as well, right? But this looks very... Did you know this company before as well? Did it also surprise you? Yeah. Okay. Cool. What's next? We have...

Bart (08:59)
and I can sit on his neck and can fly me places. Yeah.

No, you're not. ⁓

Murilo (09:23)
Cogni. So Cogni is an open source quote unquote memory layer for AI agents combining vector search with a graph database to keep knowledge both searchable and linked. It swaps classic drag for modular ECL pipelines for self and self hosting aiming for durable context with minimal code. Hence the quote unquote six lines promise. That's actually what they say here. Memory for AI agents in six lines of code. So.

Actually, was a medi, so a friend of ours, was looking for, I think, two options or something for his project. And then someone mentioned this, right? So the ECL pipelines, let me see where I can find it here. ECL, it's Extract Cognify Load. That's what they call it. Yeah, Extract Cognify Load. So Cogni is an application.

I think they need another, there's not just like a package, right? You also need a solution in the back, like a graph database or something. I'm not 100 % sure to be honest, but you probably need something, infrastructure that follows it as well. The idea is to give memory to your AI agents. Memory in the sense of like, if you talk to it today, tomorrow it wants to be able to remember what you said today, right? It can scale. The six lines of code is this. So like you have a...

Bart (10:39)
Mm-hmm.

Murilo (10:43)
Cogni.add, so basically you can add documents. So if you have like a book in TXT, whatever, exactly, whatever text. And then I guess you have to hit Cognify, Memify, so this is the memory algorithms to the graph. So I think this is to add a knowledge graph, so everything is graph based. And then you can actually search, right? So what does Cogni do if you have a book or if you have a lot of interactions with it? I think the way that I saw on the demo, a video demo.

Bart (10:48)
whatever text basically.

Murilo (11:11)
was that they add some tools, they add some functions, some cog knee functions as tools to an LLM. So the LLM can also say, add this to the memory graph basically. ⁓ Or search this from the graph, right? And then when you interact with it, can, this is actually a little image, let me see if I can play real quick. That was also an image of the graph that they show, right? So that's why they say it's also graph rag.

Bart (11:22)
Hmm.

Murilo (11:37)
So it's just something to kinda, they call it to add memory to your agents, which I guess it's very vague, like Claude I think said that adding memory is just like editing a TXT IMD file, right? So the way that they tackle this is more on a graph search kind of thing. Yeah, this video is not playing nice with me. But it looks like something like this. So you can also query, you can also visualize the graph and see what are the entities, how is the information really dense or not dense.

The way I kinda, maybe an analogy you can make with this is also like obsidian mind maps, you know, how you have like the different articles and then they interact with each other. it's something like that as well. What do you think Bart?

Bart (12:19)
How I understand it, correct me if I'm wrong. People are probably familiar with this, if you are using chat GPT, it has the ability to remember conversations that you already had and it will pick up, like if you've already interacted with it, then it will know that Murilo is a CTO of data routes and it has some information about you. And what I understand here,

Cogni basically allows to give this memory feature to any kind of LLM or LLM based agent that you... And it's probably also more richer. It's not only based on the conversations, also you can probably point it to... These are these documents that are also important for me as ⁓ knowledge that I want in my quote unquote memory slash brain.

Murilo (13:04)
Yeah.

Yes, so

there's that. It replaces Reg in the sense that it's a graph database behind, so it's not just like vector search and all these things. But I think they also mention is they try to be in between sessions, in between code restarts, right? So if you say something to chat, or again, I think it's like if you have an application, you have a customer service spot, you talk to today, and then tomorrow you talk to it, so we can actually...

fetch the interaction that you had in the past. Even if you churn it down or something. And it's not as simple as adding everything in the conversation history because there's always a limit. So the way that they go about this is with the graph database, the graph store as well. I thought it was interesting. think it's also the first, the reason I also wanted to share, think it was the first...

Bart (13:39)
Yeah, yeah, yeah, yeah, yeah.

Murilo (13:57)
First more mature, it looks more mature, right? And the first more mature solution for graph that combines a bit like the graph databases and LLM memory, quote unquote, right? I think there was also source graph that had like a coding agent that also, because source graphs I think builds a graph of your code base, right? But I hadn't seen a lot of these things before. So that's why I thought it was interesting.

Bart (14:06)
Hmm.

Murilo (14:21)
And actually made me wonder why I don't see more graph databases or graph networks when it talks about LLM memory.

Bart (14:28)
Yeah.

And to me it's like, it's a bit of a, I should put it a bit of a vague boundary between memory and knowledge. Right? Like because to me, purely memories, like I know that I had this discussion with you already. Like it's not like, I'm also going to inject like, this is my quote unquote pre-existing memory or knowledge for everybody that I started conversation with. Like these are documents like with you.

Murilo (14:51)
Yeah.

Bart (14:51)
typically even in the rack system, But I understand if I look at the documentation, not sure if I'm correct, but it combines these two a bit. Okay.

Murilo (14:53)
Yeah, indeed.

I think it does,

I think it does combine. Again, I'm also, I need to dive into more, but I think they would also have, like I shared, in the examples I saw, to be honest, it was only one graph, right? But I would imagine that if you're actually building something, because they do mention interacting with customers, right? Like,

Bart (15:17)
Hmm.

Murilo (15:18)
If

I talk to you today, like if you build a chat bot for like ⁓ an application that we have, you want that bot to remember the previous experiences that it had. So I wonder if there's like a bit of like two graphs that like one is a shared one and one is more like personal to you if you can set it up on the database level. But yeah.

Bart (15:37)
question.

I'm also interested by this, how they call this pipeline, like an ECL pipeline, extract, coctofile load. It's a bit of a different way of thinking about ELT basically, or ETL maybe, maybe more around there.

Murilo (15:44)
Yeah.

Yeah, it did.

Exactly. Yeah, ETL, ETL.

Yeah, I did. I also thought like, but it says so, and this is from the first sentence on the reading, right? Like, use your data to personalize dynamic memory for AI agents. Cogni lets you replace RAG with scalable and modular ECL. So extract, cognify, load pipelines. So yeah, again, they also put a bit as like a replacement of RAG, right? So.

Bart (16:17)
I'm wondering what's...

for what use cases this becomes crucial. Because it's also like, it's complex to have. Like if it would just be like a customer support, conversation history, I mean, that is in the end, like you can simplify it as an TXT file, I could become like, I'm wondering like what you kind of use cases you would definitely go for this. And I also think like if there are a lot of use cases where this becomes relevant, like it's a matter of time that's.

Murilo (16:33)
Yeah, for sure.

Bart (16:44)
like one of the big providers like OpenAI or Entropic or whoever like comes up with their own service offering around this.

Murilo (16:51)
Yeah, that's actually true. But I think that's the case for most LM application services, right? That's true. I'm wondering, I'm thinking a bit with you.

if you have a chat bot to help you buy things maybe. So you remember what are the past things that you wanted to buy, how they relate and how you relate it to those things as well. I'm also, mean, I don't know. I think my mind goes a lot to like networks, right? So like social, I mean, even the examples I think that was like a social network kind of thing. know, like I've talked to this person, I send this a friend request or like if Meta has like a little chat bot of like create an event and buy to these people and then maybe you can have like these entities and go. ⁓

I'm

not 100 % sure either. And I'm not 100 % sure also even if you just replace rec with graphs, what will happen, right, like in practice? Would you get better results? Would you not? Can you do a mix somehow? Not sure. I think the thing they said, so they do have an open source and this is open source. They also have a Cogni cloud, right? So it's the same. Exactly, yeah. So I think that's a bit there.

Bart (17:51)
Of course.

Murilo (17:55)
their business, right? And there was some awards as well on the repo, right? The product of the day and the repo of the day. ⁓ Got me curious, got me curious. I don't have a use case for it either, but maybe I just need to be a bit more creative, right?

Bart (18:05)
Nice.

Murilo (18:13)
What else?

Bart (18:13)
⁓

Python is set to add explicit lazy imports, letting developers defer module loading until first used to speed startup and cut memory. The accepted proposal targets Python 3.15 and cites potential 50 to 17 % startup gains, while keeping normal inputs unchanged for backwards compatibility. Explicit lazy imports in Python. What is this, Marilla?

Murilo (18:38)
Yes, so PEP is a Python enhancement proposal. basically this is anyone can create these things, right? So basically they want to add a new feature to Python. In this case, it's a big feature because it adds a new keyword. Before I talk about what the actual change is, lazy imports, as I understand basically is usually in Python, the best practice or clean code, right? You add your imports in the beginning of the file.

But that means that every time the file is run, the first thing that it does is to import everything, which is considered good practice. But that means that if there is a dependency, something you're importing the beginning of the file, but you don't use it, or you only use when there's one function that you need to call, that means that you need to pay for that cost of doing the imports in the very beginning, even though you may never actually use that function, right?

Bart (19:24)
And now maybe other example just to help understand why this can be a problem. Like if you say import ⁓ my AI helper and that AI helper then loads a big model into memory, which takes a long time, a few gigabytes. Like it will slowly, like it will, you will import it at the start of your, of your application. It will slow everything down basically.

Murilo (19:45)
Yeah, exactly. I actually, maybe a question as well. If you import that file in many places, does that get loaded many times? Normally not, right? Yeah. But I know that there's a lot of people that have actually gained a lot of speed. think one that I remember very clearly was a Python package manager before UV. It was called Hatch.

Bart (19:56)
No, normally not.

Murilo (20:10)
Hatch also use lazy imports. So basically just import inside functions, right? So whenever the function is called, do that. But even if there's two functions, they use the same import, you need to duplicate the code, yada yada. But that's basically lazy imports. You import it exactly before you need it.

And the explicit is import to this enhancement proposal for Python 3.15 and which actually was already accepted, which means that Python 3.15 will have that. You have a new keyword that you can add before import. So if you type, instead of typing import JSON, you just write lazy imports JSON. That means that Python won't actually import the package until it actually needs to use it. And everything happens behind the...

behind the hood, right? Under the hood. So, or if you have like lazy from JSON import dumps, then there you go. And then they also show.

Bart (21:01)
and it will only be actually

imported the first time that you use it instead of at the beginning of the script.

Murilo (21:05)
Exactly.

Yes, yes, yes, exactly.

Bart (21:09)
And people,

like you were explaining, people ⁓ had some workarounds before this, like by having your import statement in a function and then the first time the function gets called, only then the import runs. But I think it's considered a little bit unpytoning in a sense that you don't have a full overview on all your dependencies at the beginning of your file, basically.

Murilo (21:19)
Yeah.

Yeah, I'm also wondering if you can, because if you have that and you have like circular imports as well, you won't find it out until you execute the code, So, yeah, if you have lazy imports in the...

Bart (21:38)
With lazy, you mean, lazy import.

Yeah, that's good question. I don't know if there are any checking utilities for that. That's good question.

Murilo (21:48)
Yeah.

So yeah, mean, they also had some syntax errors, right? So if you've tried to lazy import inside a function, so basically try to do what people do before with lazy imports with the actual lazy keyword, then there's going to be some syntax errors. But in a nutshell, if you have a lazy import JSON here, and this is an example I'm showing you on the screen, you only see actually the...

the module inside the important packages after you used it. So, which could again improve the performance of a lot of things, right? So, I it was pretty cool. I also saw some, tell me.

Bart (22:18)
Yeah. Maybe also,

maybe also interesting is that it didn't follow the discussion on this ⁓ proposal at all. but, like the, was created the second of October and it was already, approved the 3rd of November, which is, ⁓ very fast. So I think there is a, a strong consensus in the community that this is something that is needed.

Murilo (22:36)
Yeah.

But think also it's who wrote it. I think Pablo Galindo Salgado, he's really big in the Python community. He's Spanish and he was also a release manager for previous Python versions. So I think because he also wrote it, he probably already pings some people, like there's already some noise in... Yeah, yeah, yeah. Yeah, exactly. So yeah. know, a pep that actually took a long time was the...

Bart (22:56)
Sure, yeah, yeah. He did the politics.

Murilo (23:05)
lock files in Python. That one took so long. Like there were so many versions of it. And one thing as well, like I was talking to people that maintain this and maybe this is not news to you, but when I first heard it, I was surprised that basically there are these PEPs and then they get accepted and people work on this basically. So basically the PEP should just describe how the thing should go and then someone actually goes and implements this and merges into the C Python repo.

But just because a PEP is accepted doesn't mean it gets implemented. So there's no higher, no one can make anything, yeah, no one can force anyone to implement anything. And also things can get merged in that there was no PEP technically, right? So if you are a core contributor of Python, you can always push.

Bart (23:35)
Okay, interesting.

Hmm.

Murilo (23:48)
you can always merge code into all these things in the main Python repo. But normally when this happens, and that's what I heard as well, that people are quick to refer to these changes. So basically the community is self-organized, but there is no hierarchy between who accepts these steps, which today is the steering committee, and actually what gets written up and deployed. I thought it was interesting.

Bart (24:09)
Interesting.

Murilo (24:12)
Pum pum. Anything else here, Bart?

Bart (24:14)
⁓ no, not really.

Murilo (24:16)
Would you use lazy imports?

Bart (24:18)
Definitely. Yeah, I've had situations in the past where I had imports and functions just because it was way more efficient and this makes it simpler.

Murilo (24:19)
Definitely.

Okay, cool. What was the case actually that you needed performance, that you needed to lazy import stuff?

Bart (24:34)
Oh, multiple ones.

For example, I forgot what the exact thing was, but it was a library that had support to read multiple types of files. And I don't know what I used anymore, but for a certain file type, maybe I used pandas, for another file type I used parquet. And like you had things that were only...

like using specific scenarios. So you would only import it if you were in that scenario.

Murilo (25:03)
I see. Yeah.

And I Pandas is a big one as well. Like, like big dependency. Yeah. Okay. Cool. Cool. Next we have the International Criminal Court will migrate from Microsoft Office to Open Desk, an EU backed open source suite, an emblem of Europe's push for digital sovereignty.

OpenNASK is built by German Zendes and tied to a new EU level initiative amid lingering tensions over dependencies on US tech.

Bart (25:34)
Yeah, I thought this was an interesting news article. the ICC is the international.

criminal court.

Murilo (25:42)
Yeah, I think that's what it is. International Criminal Court,

Bart (25:45)
Let me just verify that. International Criminal Court, yeah. Phew, we don't need an edit. So they're switching away from Microsoft Office basically to OpenDesk and I never heard of OpenDesk. I think that was one interesting one. If you go to the website of OpenDesk, opendesk.eu.

Murilo (25:46)
No it isn't, it's okay.

Yeah.

Bart (26:01)
It shows all the type of applications they have. It's basically like the office suite but an open source alternative. You have calendar, have chat, have docs, have email, file storage, notes, projects, tasks, video calls. It sounds like quite a complete suite. Never tried it. I can't give any opinion on that. So it's interesting to hear these movements too. Let's say more.

Murilo (26:21)
Yeah.

Bart (26:27)
sovereign tool stack especially for something that is very

for something that should be probably sovereign, something like the International Criminal Court, right? And linked to that, what some people have hinted to, this might also be a reaction to the geopolitics on this, because the Trump administration, they...

Murilo (26:35)
Yeah, that's true.

Bart (26:51)
Put sanctions on members of the ICC already in the first term of the administration. They were reverted by the Biden administration. They're now again, their new sanctions are placed on ICC members ⁓ due to ICC stand on Israel, stuff like this. Let's not go into details.

Murilo (27:08)
But then the sanctions were like from Microsoft or like the government somehow? Okay.

Bart (27:11)
No, no, they came from the US and

basically all the people of the ICC, took a visa in the first example. But it shows a bit like something like the ICC should not be pressured into...

tools that are, in the end, you can put export restrictions on this. If you're heavily dependent on Microsoft, you don't have a good relationship with the US government. There is a potential thing that they put, when they put sanctions on you, that there are export controls, that you're not allowed to use these things anymore. It's all, all of this is very, it sounds far-fetched, but when you're such an organization, there is a good reason to...

Try to be as autonomous as possible, right? Because otherwise you might be biased in whatever you do.

Murilo (27:58)
Yeah, yeah.

Yeah, and I think you say it sounds far-fetched, at the same time, Trump administration is also... Exactly, yeah. It's near-fetched. It's... Yeah, exactly. It's just fetched, right? It's just there. Yeah, I think they said that earlier this year on the article as well, it that it was reported, so Microsoft rejected, but it was reported that Microsoft had also suspended...

Bart (28:05)
These days there are very little that is for fetch, right? Every day there is news saying, what? Like, no, this can't Yeah. Yeah.

Murilo (28:26)
his email account as well. So like an administrator from an ICC official, right? So, I mean, again, maybe, yeah, maybe it's not true. It was rejected and all these things, but I do think it makes sense. One thing I was a bit surprised, because I never heard of this. I guess this is, because when I thought of Microsoft Office, I thought of like LibreOffice, right? But I guess this is, yeah.

Bart (28:47)
Yeah, yeah, that's also what comes to mind.

Murilo (28:50)
But this is also much, much broader than that, right? Like I think there's a lot of services here. Cause I think LibreOffice is really just...

Bart (28:57)
This is not

necessarily an Office replacement, it's more like a Microsoft 365 replacement. With a Teams alternative, a context alternative, user management.

Murilo (29:02)
Yeah, exactly, exactly. which is more impressive, right? There's a

lot more here. There's a lot more here. No, it's really cool. I wonder, like in the website I'm showing you on the screen for people listening, the website looks nice. It looks modern as well. It doesn't look... ⁓

Bart (29:19)
I'm

wondering if in ⁓ Belgium or Flanders they've also explored this. We'll have an interview probably in December with representative of Flanders government. We'll make sure to ask her.

Murilo (29:26)
You're right.

Mm-hmm.

Yeah, we're gonna put down the question there, And actually, how does the, this is open source, but is it also operating models? Can you, like, you hosted yourself or do they have a cloud offering as well? There we go. You can deploy in your own data center, yeah, SaaS and confidential cloud, okay. Looks cool, looks cool. Again, I'm very curious how, how...

Bart (29:37)
Yeah.

I think it's both, right?

Murilo (29:56)
how this works, because also a lot of people are not super happy with Microsoft 365 stack either, right?

Bart (30:02)
Yeah, there's a lot, maybe another debate, right?

Murilo (30:04)
Another debate, yeah, yeah, yeah. I'm just, yeah. I will say though, sometimes teams, for me, it feels a bit bloated. Like...

Bart (30:15)
Teams, I'm

not a team sound here.

Murilo (30:17)
It takes so much memory. Yeah, tell me. I feel like there's a hot take coming.

Bart (30:17)
I do think that

especially like Excel, is no alternative to Excel.

Murilo (30:26)
Yeah, that is true. Excel, think, is the one thing that you really, there is no competition, right? So, yeah, it's true. But they have a spreadsheet solution here.

Bart (30:35)
Yeah, but there are like tons of spreadsheet solutions and it's like, it's always like, hmm, yeah, hmm. But we should try it, right? We should, there's probably a demo somewhere. We should give it a test run at one point.

Murilo (30:40)
Yeah, yeah.

We'll give it a test, yeah, indeed. What else do we have but...

Bart (30:49)
Again, news from LLMLand Moonshot AI's Kimi K2 Thinking is an open-source thinking model that interleaves reasoning with tool calls across long sequences, built on a trillion parameter MOE model with a 256k token context. It emphasizes native INT4 inference and claims stable behavior over 200-300 consecutive tool invocations.

Kimi K2 Thinking, it's a thinking version of the Kimi K2 model, was released a few days ago. And it's actually a very good model.

Murilo (31:21)
I think so. Let me check. Where can I see this?

You tried it or vibes? didn't try yourself.

Bart (31:27)
I didn't try it, no, I looked at the benchmarks.

So if you look at what we typically discuss is coding, on the SWE bench it's still ⁓ Sonnet 4.5, then closely behind GPT-5, and then closely behind that is K2 thinking, like trails only by a few points. For ⁓ reasoning and math, GPT-5's strongest, followed by K2 thinking, and only then by a clot. So that's surprising.

Murilo (31:53)
Yeah.

Bart (31:53)
⁓

There is also it actually on some ⁓ agentic tool calling challenges. It outperforms even GPT-5.

Murilo (32:01)
well, that's impressive. think especially the name of the game now seems to be tool calling, right? I mean, now if you have to choose a model to really build an agent and calling the right tools is really, it's like 90 % of the job. having a good model there, it makes a huge difference. And it's also Chinese, no?

Bart (32:19)
Yeah, that's

so Kimi's K2 model, comes from Moonshot Labs. It's a company where ⁓ like a few big backers like everybody will know Alibaba, for example, like they're one of the main investors in Moonshot. I think it's around

Murilo (32:29)
Moonshot it, yeah.

Hmm.

Bart (32:41)
for well, I want to say a long time, but everything is still very short. I want to say for last two years, but it has gone a bit unnoticed, want to say. a bit, I think in our, in the European community. think DeepSeek got lot more attention and not necessarily because it was way better or something. It uses a bit the same training approach as DeepSeek for the original K2 model. This thinking model is rumored.

Murilo (33:01)
Yeah.

Bart (33:10)
to have cost only 4.3 million USD.

Murilo (33:18)
That's it.

Pocket change, well, compared to what we see, other members we see.

Bart (33:21)
Yeah, yeah.

Also interesting is the cost. So if you use it via API. We already discussed like it's actually quite good performance, right? There are some things that only without performance. The other big players. If you look at input per 1,000,000 input tokens, so KimiK2 thinking cost 60 cents. GPT-5 is $1.25 Claude Sonnet 4.5 is $3.

Murilo (33:30)
Yeah.

Bart (33:46)
So it's like a fraction of the input token cost. For the output token, the difference is even bigger. So it's super cheap compared to these other providers. So it's very aggressively priced, right?

Murilo (33:53)
well.

Yeah, I think the...

Bart (34:01)
And I think

what is also interesting, like it's an open weights model, so you can actually download it and run it yourself.

And that's... like...

Murilo (34:08)
and you have you ever done it like

how how how how feasible it is though to to run these things

Bart (34:13)
Well, I think

for the average hobbyist at home, it's not feasible. ⁓ But if you're, let's say we were discussing the ICC, if you're like a large organization and you want an in-house model, I think it's definitely feasible. To me, also shows a bit, like the LLMs itself, they've become a bit a commodity. Like you have actually, like which we something that we didn't expect.

Murilo (34:18)
Yeah.

Not true.

Hmm, yeah.

Bart (34:36)
to see at all two years ago, you now have like literally open weights models that more or less compete with the latest open AI and entropic model.

Murilo (34:39)
Yeah.

Yeah, indeed, true. That's true. do you think this is... So the Chinese models, as far as I know, they are open weights, no? Like I think there was Quan, Deep Seek, but like the big ones that I remember is like Kimi, Quan and Deep Seek. Or am I forgetting another one?

Bart (34:57)
A lot of them, not all, but...

Yeah, I think that's the most

well known. We actually discussed another one, but I forgot the name. I think those three are the most well known.

Murilo (35:08)
Yeah, I think me too.

And they're all open, open weights, right?

Bart (35:13)
They're

all open weights and there are actually some rumors that...

China is also doing this very strategically. Like there's a lot of rumors that these are also government-backed funding models behind to this. That is also a bit of a way to basically dump AI tech in the global market to upset the competitive landscape basically, because which is the US? And I think if the reality becomes that LLMs will become more or less a commodity.

Murilo (35:26)
Yeah.

which is US.

Bart (35:45)
I think at that point, like the only difference you can really make is like the tool set around it. But it's way harder to compete on that, right?

Murilo (35:54)
Yeah, he did. Yeah.

Bart (35:56)
And I think if China

really does this like this is a dumping mechanism, but we just dump AI tech on the global market, this might be very hard for the US, right? Like the US economy, if you take away all the AI companies, it hasn't grown. And in some domains, actually negative.

Murilo (36:14)
Interesting.

Bart (36:15)
because it looks like on the stock market a very good year, but it's only driven by AI related companies.

Murilo (36:19)
AI. Yeah.

that's interesting, but it's not so surprising. I can imagine how that's the case. And do you think, personally, do you think this is the strategy? I mean, I think it's hard to say, right? I think for sure they thought about it when they made it open, right?

Bart (36:36)
Well,

you can also look at it more from a, because this sounds very aggressive, right? Like if it's really to upset the current global economy. But you can also say like, like China in the quote unquote Western world, doesn't, historically doesn't have a very strong name when it comes to really like having a very, let's say trust relationship. Like we trust these companies to serve us and ⁓

Murilo (36:43)
Yeah.

Yeah.

Bart (37:00)
So they have to overcome this. think because of this, it's hard for them to create a new OpenAI or a new Anthropic. But by dumping these technologies as open source, everybody can use it and everybody is using it. Because everybody that is today running an enhanced model, there's a good chance that they're using QAN, for example. QAN has been very big, it's very easy to retrain and to fine tune it.

Murilo (37:17)
True. True.

Bart (37:21)
And like it builds this trust because we are building on these open source technologies and like they're the only one, especially with now Meta, like basically downscaling their open source efforts. The only real significant efforts open source when it comes to open weight models is coming from China. So I think the reality will be that large enterprises in Europe, for example, but also like, like research institutes will use these models to

Murilo (37:32)
Hmm.

True.

Bart (37:48)
to build solutions on or to retrain or find your new models on, right?

Murilo (37:53)
Yeah, I think that they should, right? I feel like if they're not, they're leaving stuff on the table. Right?

Bart (37:58)
⁓ And that

does help like build that trust relationship again.

Murilo (38:03)
Yeah, that's true. That's true. mean, if anything is definitely bringing attention to China, that they can build really good AI models, right? Like I think it's... I think if you ask the beginning of the opening, I mean, before DeepSeek really, or maybe not really before DeepSeek, but like not that long before DeepSeek, I think people weren't looking at China for this AI expertise, right?

Bart (38:09)
Exactly, exactly.

No, they were looking at just

the menu. People were just thinking about Teemu and that you could buy stuff for cheap there. But that didn't link to their quality and high tech. If you want the last state of art, you need to look there. You didn't have this reflection.

Murilo (38:29)
Yeah, exactly. So no, switch.

Yeah, exactly.

And I think today that's the case for sure for our lens, right? One thing also I saw, I wasn't going to bring this article up, but I also read a bit. since we talked about Kimi K2 thinking, basically the guy had like five thoughts, right? So open models release faster. So he was saying like, he feels like these open source models are being released at a faster pace than OpenAI and quad.

Bart (38:44)
Yeah, big change in last years.

Murilo (39:06)
I think...

Maybe the thing for me is like, because it says release faster, I think when you release faster, the expectation is not as high as well. So I feel and maybe that's also something. Maybe the open AI models or the cloud, I don't know this as much, but I know for open AI, there was also some backlash, right? That the GPT-5 was, there was a long wait before they released it. And when they did, some people were disappointed.

Bart (39:25)
Yeah.

I think you can argue that OpenAI and Anthropic are still a lot bigger than the companies behind these models and that they also have a larger user base and that they need to do some more rigorous testing so that the experience stays good. I mean, don't think it's as straightforward, right? To just simply say like they are faster or slower to release a new model.

Murilo (39:47)
I see, I see. The other thing they mention is to, there's a lot of vibes, right? So user behaviors, but they say like they are very strong on the benchmarks. ⁓ And that's probably what the Chinese are also, maybe also in the speed, right? Like they look more a bit at the key benchmarks, which is also maybe the way to go as well. He talks about China's rise, which I think we already talked about.

Bart (39:56)
Mm-hmm.

Murilo (40:09)
One thing he talked about as well is the thinking with many tool calls. So I don't know if actually the other models works like this, the reasoning models from OpenAI, right? But it's a bit of a, this mix between the two things that seems a bit novel at least. And the last thing, and that's what I think we're discussing. That's also what I wanted to ask you. Like, do you think there is a lot of pressure on the American labs, like OpenAI and Tropic and Gemini?

Bart (40:22)
Yeah, yeah.

I think definitely, yeah. I think that's what I'm saying. I think the LLM models themselves, they are getting closer to being a commodity and like interchangeable. And how you need to compete today is like having one of the best models or at least like outperform in a specific domain like like entropy does with coding. Plus if like the whole

Murilo (40:35)
Do you think this is a...

Yeah.

Yeah.

Bart (40:56)
tool chain around that to make it usable for the type of customer you're aiming at, right? Like if it's B2C or B2B, like you need to really go the next mile on everything around the LLAM model, not just having the best LLAM model.

Murilo (41:10)
Yeah,

yeah, yeah, true. I'm also wondering because OpenAI also is putting, OpenAI has a lot of revenue, right? But not necessarily, like I heard that OpenAI has still like big, big, big debt and they're actually not making a lot of profit. It's just that they, I mean, they make money, but they burn a lot of money as well, like with all the data centers and all these things. I'm also wondering like,

if it's like a risky bet, you know, because they see these like open models coming and like if it becomes a commodity and they don't have both as well, you know, I don't know. I wonder if it's a tricky situation for them.

Bart (41:44)
Yeah, the whole OpenAI being over committed is maybe another story, OpenAI is saying that today, the most recent statement of Simon Holtman is that they're 20 billion in ARR, so annual recurring revenue that is, but at the other side, they have committed to invest 1.5 trillion in data centers.

And the world is questioning how can you pay for this in the coming years if you're only doing 20 billion and there are plans to to multiple million of... hundreds ⁓ of billions of dollars over the coming years. It's still far from being able to pay for 1.5 trillion in commitment.

I think that is also why we today have a lot of discussion on is there a bubble or not that will pop. At the same time, I also think it's a strategic investment and that's also what you see. This is then maybe everything around NLM because what OpenAI is doing is they're buying a lot of compute power and I can very well imagine that.

Murilo (42:39)
Yeah, sure.

Bart (42:55)
companies in the future when you are a big enterprise that you're not just buying access to the model, but also buying access to your own compute power, which definitely not everybody has, right? But which openly I will have in the future. So yeah, let's see how it unfolds the AI bubble.

Murilo (43:06)
Yeah.

Yeah,

that's true. And like you said, OpenAI is becoming more more, more than just an emortal provider, right? They're starting to do more things as well. So to be seen, to be seen. Next we have the FBI obtained a court order compelling registrar Tukos to hand over customer data tied to archive.today, the web archiving site often used to bypass paywalls.

Hayes says, the document's authenticity isn't verified, spotlighting murky jurisdiction, the operator's identity and possible chilling effects on archiving. So archive today Bart.

Bart (43:48)
So Archive today does a few other TLDs as well, I archive.is, I think there's another one. What we're seeing here is that there is basically, the FBI is looking into it. So what does it do, archive.today, is if you have...

Murilo (43:54)
I think I saw that song.

Bart (44:03)
a link to a news article that is behind the paywall. Someone that has a subscription can basically say to ⁓ Archive.today, please save this snapshot and then everybody else can just look at it without having to go ⁓ through the paywall.

I and probably you as well, we look at a of articles and a lot of times there is a paywall, so I won't confirm nor deny that I sometimes use this. But for certain cases it's very interesting. I think the concern from the community, and because I think there's a lot of uses on this, the concern from the community is that if Chucoast, which is the provider of the, which is basically behind the...

Murilo (44:32)
Hmm.

Bart (44:46)
think the archive does today domain. If they give the information on the owner to the FBI that no one really knows what this means, right? Like if, if this person is found and whatever that means for the person.

But we don't really know if there is a person behind it, or if a team behind it. If this person falls away, does it still exist? What is the impact on the other domains? There is also a union domain, is it linked or not? If this person falls away, does the infrastructure still exist or not?

like brings a bit into question what the longevity of the platform is. And no one really knows because it's very vague, because also on very gray legal grounds, of course, is holding. So it'll be interesting to see how it unfolds. What I also found interesting, following the discussion on this a bit on Reddit, but also on Hacker News, is that you have...

Murilo (45:26)
Yeah.

Bart (45:36)
A lot of people like open arms saying we freedom of information, this belongs to the people, this belongs to the global population, which I think is fair, right? Like I wish everything was open. But at the same time, like in another situation,

Like it's probably the same people that are saying, open the eye or entropic should not be allowed to just scrape everything and just take everything because there's copyright. And like, it's, it's really like, it feels a bit like if it's like the Robin hood for good thing, then it's okay to do these things. If it's a big corporation, then it's against the people. Right. And it was very interesting to, to, read this. ⁓

Murilo (46:01)
Yeah.

Yeah.

Yeah, yeah, yeah, exactly.

Bart (46:17)
disc scores on it, because like I say, it's a completely different disc scores than you have with these copyright cases on these big corporations.

Murilo (46:25)
Yeah, yeah, true. Well, yeah, feel like just because no one's profiting from this is like people say, it's okay, we should, we should allow this, right?

Bart (46:34)
Yeah,

maybe that's the significant difference here. That is a fair point, yeah.

Murilo (46:38)
Yeah, but you know, I listened to a podcast and there was one guy, I don't remember who it was, but it was something about archiving the internet as well. And I think he explained a bit why he started to use it. I think he said he was in China and I think they started archiving the internet because in China there's a lot of censorship. So sometimes there was an article and literally like you wouldn't be able to read it later.

Bart (46:54)
Hmm

Okay, interesting.

Murilo (46:59)
So I think he had like a Chrome extension to really just like start archiving stuff. And they talk a bit about like why this is also important, right? Cause that's also my question. Like, okay, today maybe people, maybe the most valid this bring is that people use it to get past paywalls. Which arguably it's already wrong, right? Because you should pay for the author's work. If this goes away, what are we losing?

Why should we, aside from the paywall thing, why should we fight for an internet archive?

Bart (47:28)
Well, I think the case that you discussed, if you're living in a country where there is the big firewall and you're blocked from everything, I think the argument doesn't really fly there because they will probably also block Archive. today, right? It doesn't give extra access or extra view on what is happening in the world.

Murilo (47:42)
Yeah, true. True.

Bart (47:46)
I find it a hard one, to be honest. But maybe I'm not thinking from the right point of view. I'm thinking from my own point of view and I see why for me, and especially in my situation where for this podcast we're looking at a lot of different news sources every week, like it would be...

Murilo (48:03)
Yeah.

Bart (48:05)
wouldn't be possible to pay for all these things, right? Like, we wouldn't be, like if we would have to do it, we wouldn't be making the podcast because it was just not realistic.

Murilo (48:12)
Yeah, yeah, true, true, true. Do you feel like the almost looking at the internet history as inherited somehow, like how websites they were before and what they from do you find almost like a big, big public library, everything that was written? mean, if I mean, it's never everything, but like, imagine that there were retro websites or some things that like you

like to have recorded somewhere do you see value in these things as well or maybe less than other

Bart (48:43)
Well, you're

basically referring to archive.org, right? Which is like an official... ⁓ The actual internet archive. It's not used to buy bus paywalls, but you can archive basically any website and it does a lot of automatic automatic crawling. I do think there is a lot of value in that, to be honest, especially when you look at it long term, like... I think...

Murilo (48:47)
I'll come to them again. It's true.

Bart (49:05)
the way that people...

Remember history is very flexible, very influenceable also by the messages that you give to people in the present. And I think for therefore it's good to have an archive of everything that is written down, of as much as possible of that is written down. And today a lot of what is being written down is digital. And to me, the internet archive is like...

Murilo (49:27)
Yeah, indeed.

Bart (49:30)
an extension of the physical library network that we have across the globe, right? So I do think that there is a lot of historical value in that.

Murilo (49:35)
Yeah, it's true. It's

More for the actual archive, but then maybe this side satellite archives.

Bart (49:46)
Well, I think

the archive.today, which again, you can use it to bypass paywalls, it gives access to information that is not gonna get archived on the internet archive. I think you can make the same argument that it's super valuable to have this stuff that is written down, if it is behind the paywall for accurately reflecting what happened historically. The problem here is like...

that there is a legal and copyright component to this.

Murilo (50:12)
Yeah,

true, true, true, Yeah.

Bart (50:14)
But I also do

think, like, we say that it's at the corporate level, at the commercial level, we say, fuck everything and then dropping an open eye and everybody, like, they can scrape the whole world and so be it and they can profit from it, then this should also be possible. I don't think that there should be a discussion on that.

Murilo (50:28)
yeah, for sure, for sure, for sure, for sure. And

I think it's also maybe the last thing I'll say on this as well is like, if we had a subscription and we share the subscription, is it, because in some ways like there is someone that is paying for it and they're kind of sharing the content, they're choosing to share the content with internet.

Bart (50:45)
Yeah,

and maybe you can there like make you're saying you have a subscription to the Financial Times and you share this with others via the archive today basically, right? Like you share the content and you can maybe make the parallel there like with the the anthropic buying a book and training on the book and like everything that's created because of that like a shared world like and they profit off it. ⁓

Murilo (50:55)
Yeah, exactly.

Yeah.

Exactly. Yeah, Yeah, indeed. That's

a bit...

Bart (51:10)
I think the whole

big difference with the legal aspect here is like these big L.M. operators, they have the money to buy, to basically put legal advisors on this and drag this along for a long time and at certain points settle. Like the person behind Archival.today, if he tomorrow gets a big fine or he's jailed, like it stops to exist, right?

Murilo (51:31)
Yeah, that's it, game over. Yeah, it's true. It's true. Which also feels a bit more, even more unfair because this guy's probably not profiting from this, whereas the big LM providers are.

Bart (51:42)
I can imagine someone that is behind archive.odesa that's really out of a fundamental belief that there should be freedom of information.

Murilo (51:50)
Yeah, exactly. Yeah,

like a big guy with like a big beard, long hair, not a lot of sunlight, you know, it's like for the people.

Bart (51:59)
Probably like a Wikimedia sticker somewhere.

Murilo (52:02)
Yeah, yeah, exactly.

All right. What's next?

Bart (52:04)
Hucking Face's small team published a hands-on playbook distilling what actually worked in training small yet competitive LLMs. From data curation and loss curves to post-training and reliability, it reads like field notes for practitioners with checklists, pitfalls and system lessons that invite debate about trade-offs.

Murilo (52:24)
Yes, so this is from the Hugging Face team. I saw it coming on LinkedIn from the CTO of Hugging Face, I wanna say. So it's the Small Training Playbook, Small S-M-O-L. it's like their brand, guess, on whole everything in LM, right? They also have small agents and all these things. And then it's called, the Secrets of Building World Class LMS. So he did mention, like in the LinkedIn post, it was mentioned as an ebook, kinda.

But if you click, it kind of shows more as a blog article, I guess, or like as an article, like a very long article. And basically this is everything that you would need to know, maybe a lot of asterisks, if you want to train your own models.

Bart (53:13)
Interesting.

Murilo (53:14)
So they talk about fine tuning, the data, the importance of finding the right data, the different strategies, the different model architectures. So there's a lot, a lot, a lot of stuff. I didn't read the whole thing, right? But I thought it would be like, it's an interesting.

Resource for people that are curious on how these models get set up We also talked about how China has been getting really good at training these models as well and opening the weights So this is also a bit of that Yeah, it's a bit of like comprehensive playbook so they even talk about like the infrastructure Exactly. Exactly And I think we are in a place now that we do have more

Bart (53:46)
it democratizes a bit like how this gets made and

Murilo (53:56)
people coming in training their models and like sharing a bit more how they're doing, right? So I feel like there is a bit of a convergence right up on the architectures and how to go about all these things, right? But even like the infrastructure, like what kind of GPUs you want, how much you can expect, how much data you need to get a good model, finding the right data sets, right? Post-training as well if you wanna fine tune you later. I would imagine even things like...

Bart (54:11)
Interesting.

Murilo (54:18)
how to run the inference as well, what kind of infrastructure you're looking for. They also talk about reasons why you should not train a model. So things like we have computer available, everyone else is doing it, we want the best model possible, or AI is the future, they're not good reasons to train your models, right? To train your own models, let's say. But again, they kind of go on and on. I think the one on the architectures, I also thought it was quite interesting, but I need to find it as well.

Bart (54:20)
Hmm.

Murilo (54:44)
but they basically talk like mixture of experts, what does it mean, the dense networks or like all these different things, right? So I do think, I do expect this, so again here, they have architecture choices, they talk about attention, embedding, sharing, stability, so stability of these models, the going sparse, the mixture of experts, excursion, so there's a lot of stuff and I think I saw also in the introduction that this is a bit the...

Look behind the scenes of training small LM3, a three billion multilingual reasoning model trained with 11 T tokens. T I guess here is thousands? don't know. Trillion? I guess it's trillion. Yeah, it should be trillion. So yeah, so I think they kind of basically did something themselves. I think they probably did their research to have like a lot of authors here. And then the...

Bart (55:20)
true?

Murilo (55:32)
And then just kind of publish this as an ebook, kind of.

Bart (55:34)
Hmm.

It's important

to have these information sources available.

Murilo (55:41)
For sure, for sure. And I think Hugging Face, they already had some stuff. again, it feels thick, right? So I think I would expect you need to have already some background in the mathematics of AI and how these things go. I do think it's very technical, but I do think it probably has a lot of interesting...

Bart (55:52)
Hmm.

I

think what's interesting is that there is this combination of how does everything work, but also how do you apply it? And I think the latter, like how do you apply it is very time sensitive. there are different difference ⁓ framers, like the hardware evolves very quickly. So I think this combination of things is very interesting to have available to the public at large. ⁓

Murilo (56:10)
True.

That is true, that's very

true. I imagine that like next year, it will be very different. And maybe even now, right, like someone can read this and say, we could also make a package that can take care of 30 % of these things, right? Which actually, there was some things from Microsoft that also mentioned this.

Bart (56:31)
Exactly.

Maybe also a shout out to Karpati that did something really similar. It's not really a guide, but it's a very simplified implementation of something that looks and feels like Chess GPT. It's called NanoChat.

And it's basically like a full stack implementation. it does the tokenization, pre-training, fine-tuning, evaluation, inference, web serving. So you also have a simple chat interface. So that's really like from nothing can end up with a chat GPT-like interface. If you really want to get into the nitty-gritty of how this is done, like you can just...

try to understand this repository and try to re-implement it for yourself. think these types of learning resources are very important.

Murilo (57:22)
Yeah, I think it's also cool to kind of... I think that's also the cool thing about programming in general. That's one of the reasons why I enjoy programming so much. It's like, it feels... And I think for a lot of times maybe it's a bit of an exception, but if... I did mechanical engineering before, right? So a lot of the stuff you learn is like, okay, you learn about it, but you can't really test it out.

Bart (57:39)
Yeah.

Murilo (57:40)
You can't

just go ahead and just do it yourself and then really look under the hood and really see how the cheese is made and stuff. And I think for programming and LLMs, okay, there's a big cost for the training, for the infrastructure.

But there's still like these things, right, that you can actually go ahead and look at the code and everything's available to you, right? There's no secrets here. You can really kind of go as far as you want, really, right? You keep digging. So yeah, it's really cool. I had actually seen in passing the NanoChat as well, but I haven't really looked that much into it as well. Question, do you think that the future is people training their own models, actually?

Or maybe the better question is how long in the future, how long will it take before people actually start thinking, I need to change my emails now.

Bart (58:31)
I really doubt that you will ever do this as an individual. But I do see, especially with all this information being out, with computers becoming cheaper, not being cheap, but cheaper, with also a lot of bass models available, open weights that you can start from. I do see large, maybe also...

state-backed organizations building their own models. Like we've seen, for example, examples of European initiatives to have a sovereign LML, like these type of things. I can see this happening, for example, also at large financial institutions. ⁓ But I think today it takes too much time and effort and expertise to do it.

Murilo (59:06)
Yeah. So probably like the...

Yeah, so you think like maybe in the future, the big, big companies that make sense to have something very private and or like governments, right, which also makes sense to have something very, very private, right, having full ownership.

Bart (59:25)
Yeah. I think it

will take time because we've, we, they do exist these, these initiatives. you had, I forgot the name. was a European initiative. Um, it was led by a Italian research organization. Um, but when you really, yeah, it's another one. Uh, it's another one. Um, but when you look at the output, like it's, it's never really satisfying, like the performance of these things. Like I think it's still a bit early days for this.

Murilo (59:34)
from the Netherlands,

There's also a Swiss one, I think. No?

Bart (59:49)
Like I think it's not... You need a lot of high concentration of both expertise and compute power to pull this off and to build something like the new Ki-Bee model we discussed, right?

Murilo (59:49)
Yeah.

Yeah, maybe and instead of training something from scratch, do you think that there is a near future where people need to know how these things work to fine tune models? So you can take a Kimi, but you want to train it for your customers or your, I don't know. If anything is realistic, the more people we need to, we need resources like this to get these things going. Yeah.

Bart (1:00:17)
I think that's much more achievable. I think that's what we...

I think it's already been done. ⁓

I know some financial institutions that doing this. I'm a little bit skeptical these days on whether it's worth the effort because it's still a lot of effort and most likely the next big general LLM that is coming out has better out of the box performance than what you're currently fine-tuning.

Murilo (1:00:44)
Yeah, today is better to just wait for the next iteration, right?

Bart (1:00:48)
Yeah, think maybe until we get hit a bit more, diminishing returns.

Murilo (1:00:53)
Yeah.

and you don't think it's gonna come, the diminishing returns will be anytime soon.

Bart (1:00:56)
Well, this is a very debated topic, I know that there a lot of people that are saying that it's not really improving already. I completely don't agree. But it's...

Murilo (1:00:59)
Yeah.

Bart (1:01:06)
I think what is very much improving is the combination of things. The LLMs themselves are improving, the whole toolchain around it, the whole tool calling around it. The integration of how these tools communicate with each other is improving. And if you use this for coding, example, if you use Clot today versus Clot a year and a half ago, it's day and night difference, even though people are saying, yeah, but we're not really improving them all the time, but it's a day and night difference.

Murilo (1:01:34)
Yeah, sure, True, no, I agree. I it will take a while before it makes sense to really do all these things as well.

Alright, and last but definitely not least, have a pair of ex-meta designers launched Senbars Stream, a voice note smart ring position as a quote unquote, mouse for voice, with an AI companion app and discrete gesture controls. Pre-order starts at 250 or $300 with shipping planned for next summer. And the startup has raised 30 million from True Ventures Upfront and Betaworks.

So another ring part.

Bart (1:02:15)
Yeah, so even though I lost mine, we are both ⁓ smart ring users. We both have an aura. That's why I put it on there. I'm... Before I lost it, I was a fan of my aura ring.

Murilo (1:02:21)
Yes.

just like subconsciously. Okay.

It kind of looks a bit like our ring too, with the, like how the top is a flat.

Bart (1:02:33)
Yeah, to me this is not

really a design, it doesn't really look nice, This is clearly a piece of tech and they put it in the shape of a ring. It doesn't look like a ring. But I think it's... So for people that don't know, our ring is really focused on...

Murilo (1:02:44)
Yeah, that's true.

Bart (1:02:53)
like sleep analysis, performance support, like these types of things. This new thing, a stream ring, something completely different. can basically like, you can take voice notes and I can imagine that you can also use it at some point to control other peripherals like your computer with your voice. ⁓ But you can basically whisper to your ring and then it takes these notes. You can also use it, I think with your thumb on the ring to ⁓

Murilo (1:03:10)
Yeah.

Bart (1:03:19)
turn the volume up and down, to control other things. And I thought to me, like something like this, if I would have this in my O-ring, I would be super happy with features.

Murilo (1:03:21)
Yeah.

I will as well.

Bart (1:03:29)
And I... So, also the voice notes thing. I... I want to use this, but I don't use it. So when I go running and I'm running for an hour, I sometimes have like an idea and I think I need to think of this. And then I try... I have my AirPods in and then I try to start making notes with Siri and it's so shitty.

Murilo (1:03:54)
Ha ha.

Bart (1:03:54)
But

if I wear my oaring, I just keep it on my mouth, close to my mouth, and I just whisper the notes. But I think it would help me. And also something as simple as controlling voice, it's very intuitive if you can just do it with your tongue. ⁓

Murilo (1:03:58)
Yeah.

Yeah, that's Having a little bit of

like a little peripheral as well, think that could also, that could look a lot both ways. I think it's funny you say that, that I think the creator, right, is the exact same thing. A lot of ideas bubble up when I'm walking or I'm not commuting. I don't want to pull out my phone and drop it at the moment. I don't want to shout into my earbuds so where the world can hear me and talk through an idea. Yada, yada, yada, so. So yeah, seems very, very similar to your use case there as well.

Bart (1:04:15)
Yeah?

Exactly.

Murilo (1:04:38)
I wonder how the note taking works. But yeah, so they do have an app as well to take notes when you're taking notes with the ring. There is a microphone, there is like a little trackpad that you can turn the volume up, down, and a button. I do think as well if the Aura had something like this, it would be super cool. But I'm also wondering if, how, if it's possible. Like, I would love my Aura to do this.

but I wouldn't have two rings, you know?

Bart (1:05:06)
No, no, I agree. it's like there's already a lot of hardware in an hour ring. I think it's hard to also put this in probably. I would never like I would never wear probably never say never, right? I would not wear two smart rings, but I would be happy to have this as XF functionality. I would also don't think I would buy this just for these features because it's quite pricey, right? It's between 250 and 300 USD.

Murilo (1:05:13)
Yeah.

⁓

Yeah, yeah, that's 300, yeah.

Bart (1:05:31)
That's a bit steep for just taking a note every now and then, right? Maybe if I would be a journalist or something and I'm continuously interviewing people and preparing stuff and would maybe be different if I have a very specific related to my career use case to it. But now it's a bit steep.

Murilo (1:05:35)
Yeah.

Yeah.

Rob,

I'm also wondering if...

like I mean there's some cool factor I guess that it's a ring but I wonder if there are other devices that do similar things that do you need this you know like if it's really just for taking notes

Bart (1:05:59)
I think the difficult thing with smart devices

is like, it's very hard to make it like not an obstruction to use, right? Like make it intuitive. Like I think taking a note, like just by bringing your finger to your mouth, it's super intuitive. Like it's much, much less friction than like for me when I'm running and having to stop and get my phone out of my running belt. And then like, like it's, it's super simple. think that is what a...

Murilo (1:06:10)
Yeah, that's true.

Yeah.

Yeah, yeah.

Bart (1:06:24)
did very smart here, right? Like it reduces friction.

Murilo (1:06:28)
Yeah, that's true. That's true. Do you think that... It's a bit hard for me to quantify though, because I feel like on one hand, I could think to myself like putting my hands in my pocket and pulling out my phone is not that much effort. But I can also see myself saying like, it's not a lot of effort, but it makes a difference that it's just there. And I'm thinking maybe not with the phone in your pocket, but maybe for something like a smart watch.

which is something that a of people have as well. And maybe it's not that, it's probably easier, right? Just kind of like pop something and just talk on your watch. It's also there.

Bart (1:06:56)
Hmm. Well. I agree. think

that I agree with you. Like for them, the example of the note taking is just as easy on a smartwatch, right? I fully agree on that.

Murilo (1:07:09)
That's what I'm thinking. I mean,

again, I say that, but at the same time, maybe, I don't know, maybe it's bit awkward to put, I don't know, move your wrist or something. Maybe it is easier, but I don't know. I think it's cool. Again, but would I buy just because it's cool? It depends on the price. That's not right.

Bart (1:07:26)
At the same

time, like if I...

I mean, it would be, ⁓ this price to me is ridiculous with these features because it's like 250 to 300 and then a 10 dollar monthly subscription. that's a bit, I mean, for the few times that I take a note while I'm running, that's a bit steep. But like I said, if you would be able to buy this device for, I don't know, 30 bucks and it integrates nicely with your phone, I mean, I would probably do it even just for a novelty.

Murilo (1:07:38)
Yeah. Yeah.

Yeah, exactly.

Yeah, true.

Yeah, yeah, true. Just try it out. Just to show off to your friends. Maybe one thing. Yeah, right. You just buy that. Just to pretend like it is... Does your aura ring have that? Can you touch on your aura ring? I shared that actually for next week, but maybe I'll just bring it up now because someone shared this one to me and actually that's what I thought is whisperflow.ai.

Bart (1:07:59)
⁓ Especially now I lost my horror ring.

Murilo (1:08:22)
Basically, so this is for speech to text, right? This is a Mac OS app and it's not really just speech to text. It just doesn't write what you say. It's almost as if you have like a personal, how do you say, like someone that types everything you're saying, like in a court or something. What's the name of that person? that?

Bart (1:08:44)
and not your media.

Murilo (1:08:46)
Yeah, anyways, it's

like you have a personal that, right? So if you have like a lot of ums or sometimes you say something, actually, like, do you want to meet up tomorrow at three? Actually, no, five. It will, there's a little buffer that it would just, yeah, it would just write exactly what you want. And what they show, there's a video. Yeah, exactly. But very light, right? Very light reduction.

Bart (1:08:59)
Okay, nice.

there's some reduction to the actual output. ⁓

Murilo (1:09:12)
So they also mentioned how if you have, I think they probably, I don't know if they take the context of what you're writing. So they also say it gets all the names right as well. So like the name here, Sarah, Sarahage. It also gets picked up correctly with something that with normal speech to text is difficult. There are some features, why is it not playing now? There are some features as well for coding.

So if you start to mention like file types here, it integrates with the ID. So the ID already snaps the right files there. Yeah, they also mentioned there's an iOS app. The last thing that I mentioned is that even if you're actually whispering, so already thinking of people at an office, right, where you don't wanna be talking the whole time, that it actually gets, it apparently is very sensitive and it gets, well, like maybe there's a...

Bart (1:09:41)
Mmm.

Okay.

Murilo (1:09:59)
a model or something and also even for multiple languages. So there's some cool features.

Bart (1:10:05)
Interesting. And did they

train their own model or are they?

Murilo (1:10:09)
That I'm not sure. am not sure. I mean, it's difficult to say. Let's see.

Bart (1:10:14)
It's interesting.

There are a lot of these, let's say, typically based on Whisper, applications that simply do speech to text, right? But the redaction is a nice touch.

Murilo (1:10:26)
Yeah, I think so. I have,

I mean again, I have that problem that sometimes I think as I'm writing. So if I have something just to talk, it's too fast. But I did think it was an interesting in between of just like writing everything you're saying. But I have a lot, I say a lot of like or ums or something that probably don't want it to be written like that.

Bart (1:10:34)
Yeah, this is a point.

Hmm.

Murilo (1:10:48)
But it's not the same as if I just talk to a Chagy P.T. and say, I want to say these things, just write this in the text, right? It's a bit of a in-between, which I thought it was interesting, but I can say that I've tried. So I don't have any first-hand opinions here. And that's it.

Bart (1:10:48)
Yeah.

Murilo (1:11:03)
Any thoughts?

Bart (1:11:04)
⁓

No, if you're interested to follow us, ⁓ do subscribe wherever you listen to your podcast or sign up to our newsletter over at newsletter.monkeypatching.io and big thanks to everybody for following.

Murilo (1:11:20)
Yes, in the future, near future, we'll have some guests as well, like you said, you already hinted at one at the end of the year. So stay tuned. If anyone has any comments, questions, thoughts, suggestions, feel free to reach us. And thanks everyone for listening. Thank you.

Bart (1:11:39)
Ciao!