These New Neural Networks Operate At Breakneck Speed

Liquid Neural Networks could help us to achieve the next level of efficiency with AI/ML

Many of us can agree that over the past few years AI/ML progress has been, well, rapid. Now, we’re given yet another new solution to supercharge what we’ve been doing with AI and machine learning.

Some of the biggest news in the tech community involves new types of neural network models that are fundamentally different from what we’ve been seeing over the last decade.

I am particularly proud of this venture because the original team developing these solutions are at MIT’s CSAIL lab.

But even if this was coming from somewhere else, it would still be groundbreaking news that everyone needs to know about.

Let me start with an explanation of two successive types of innovative neural networks.

The first ones are called ‘liquid neural networks’ – they are able to learn on the job, and continuously process information. The research teams involved say that they are based on the brain functions of small species, like rodents and birds, and that they have four special criteria:

· Flexible

· Causal

· Robust

· Explainable

The second criterion is very important, because it explains a lot of how these networks operate with many fewer nodes than traditional designs.

The fourth criterion is also extremely important, because it preserves that idea that we shouldn’t be building black box AI systems – that we should know why they’re doing what they’re doing, while they are doing it.

Now, this introduction of liquid NNs was making waves a while ago, but what we have newly unveiled on the scene is called ‘closed-form continuous-time models,” or, CFCs.

These utilize a liquid neural network design, but there’s a key addition – researchers have figured out how to solve differential equations simulating the interaction of two neurons through synapses (applying, in the models, ‘sigmoidal’ synapse designs).

Applying differential equations to each node, these new networks can do the same kinds of advanced things that a traditional network did with 1000 or 2000 neurons. But here’s the big news – they can do these tasks with something like 19 neurons, plus a perception model. If that seems oddly specific, read on…

In a recent talk at Davos, Daniela Rus, the head of the CSAIL lab, and a panel talked about their understanding of this developing technology:

“I was really passionate about how we could build AI systems that weren’t just really accurate, but were reliable and robust enough that they could solve the most critical and most powerful problems that could exist today in the world,” said Alex Amini, an MIT scientist and cofounder of Liquid AI, a related startup. “We are really excited about this technology, because it is a new type of foundational model – it’s very powerful, and very reliable.”

Rus talked about this new approach to machine learning that makes models more worthy of running safety-critical systems.

“The end result is very compact solutions to very complex problems,” she said. Companies, she added, can deploy these models in-house and run them behind a firewall, or deploy them on edge devices.

“They’re less expensive, with a lower carbon footprint,” she said of these systems, in general.

She also talked about how these models solve for cause and effect – for decision-making, for algorithmic efficiency, how they can probe systems and explain behaviors.

“Every node is more powerful,” said Ramin Hasani, also a Liquid AI cofounder, talking about the value of compression in these systems. “You throw a lot of data at them.”

In terms of application, Hasani said the pioneers have already started making connections.

“We have pipelines, we have infrastructure; we’re talking directly with enterprise,” he said.

Going back to some of the work that teams are doing, we have the creation of ‘neural circuit policies’ based on the nervous system of the C elegans nematode (some type of cousin to the flatworm? Read more about it at github.)

In public resources, you can see fully connected, random or NCP models, and some of the code behind these types of systems.

Another way to explain this is that continuous-time hidden states allow these algorithms to work differently on input data. For example, in talking about driving an autonomous vehicle, researchers suggest that more capable networks would look not at the horizon where the road is, but at where the bushes are as the vehicle proceeds in space…

It seems to me like all of this is very interesting and fascinating work, based off of what came before – as we kept looking at neural network models, we kept finding better applications and ways to make them more efficient. But this new one is really a game-changer, and I’m proud to be associated with the same institution where these people are working to build our collective knowledge of AI capability.

(Disclosure: I am an adviser of LiquidAI.)