Testing Apple’s AI in Siri: How Well Does It Really Work?

The new Siri is already here, or at least a part of it

Testing Apple’s AI in Siri: How Well Does It Really Work?
David Bernal Raspall

David Bernal Raspall

  • Updated:

In recent years, artificial intelligence has become a key technology in the development of virtually anything, and Siri is no exception. Aware of this, Apple has been working on new technologies that can enhance the capabilities of its virtual assistant and take it to a new level.

Recently, we have learned that Apple is testing and developing natural language generation capabilities for Siri, which could significantly improve its ability to understand and process users’ messages. As far as we know, an internal team has already been testing these systems for some time, but now we can test them too.

Hidden evidence in plain sight

According to 9to5Mac, Apple is testing new natural language generation features for Siri, in a technology known internally as “Bobcat”. This new system is being tested in the latest beta version of tvOS 16.4, and could reach other Apple operating systems in the near future and also expand beyond Siri to be present in other areas of the system and support or enrich other tasks.

Right now, Siri is based on a template system, which has created significant technical and scalability challenges for Apple’s virtual assistant over the past decade. Natural language generation technology would enable Siri to better understand us and be more efficient in resolving queries and performing tasks.

The new natural language generation system for Siri is available to everyone in the beta version of tvOS 16.4. Currently, however, it is only used to tell jokes on Apple TV while exploring the possibilities when it comes to creating timers. So, all we have to do to test this new Apple AI, is to update our Apple TV to the fourth beta of tvOS 16.4 and make the request in question to Siri.

What is most interesting is that, as indicated by 9to5Mac’s findings, the code for this natural language generation is already present in the betas of the iPhone, iPad, Mac and HomePod operating systems. Although it is only enabled on Apple TV for now, it is only a matter of time before testing can begin on the other systems and devices.

The first visible step of many to come

When yesterday we read the New York Times report, which stated that Apple was testing language generation systems, we could hardly expect it to be, without further ado, directly on our devices. Tests that, although they are now closer to curiosity than to functionality, do give us a glimpse of the amount of interest Apple has in these technologies and perhaps hint at some of the timing that the company may be managing.

That said, it is important to note that the fact that Apple is testing these technologies neither confirms nor rules out that it is developing technologies similar to ChatGPT. For now and as far as we know, Apple is working on an improvement of Siri. An improvement that will allow us to interact more comfortably and more fluently with our favorite assistant.

In addition, it is worth bearing in mind that we do not have any presentation or development schedule for these features. That said, it is also true that its arrival in beta, even if only for Apple TV and only for some tasks, outlines the possibility of news and surprises at WWDC 2023 this coming June.


Add to this the fact that most of these tests are enabled or disabled at the server level, and Apple’s freedom to evaluate this technology in real-world use cases is quite large. While GPT-4 already creates applications on its own, natural language generation could be the key to Siri being able to understand us better, resolve queries and perform tasks more efficiently. The latest tvOS 16.4 beta is just the beginning of what could be a major evolution in Siri’s ability to interact with users more effectively and successfully. And, without losing sight of the power of the upcoming A17, it could go well beyond Siri.

David Bernal Raspall

David Bernal Raspall

Architect | Creator and editor of | Trainer in Apple technologies | Editor at Softonic and iDoo_tech and previously at Applesfera

Latest from David Bernal Raspall