Continuous adaptive nonlinear model predictive control using spiking neural networks and real-time learning

Model predictive control (MPC) is a prominent control paradigm providing accurate state prediction and subsequent control actions for intricate dynamical systems with applications ranging from autonomous driving to star tracking. However, there is an apparent discrepancy between the model’s mathematical description and its behavior in real-world conditions, affecting its performance in real-time. In this work, we propose a novel neuromorphic (brain-inspired) spiking neural network for continuous adaptive non-linear MPC. Utilizing real-time learning, our design significantly reduces dynamic error and augments model accuracy, while simultaneously addressing unforeseen situations. We evaluated our framework using real-world scenarios in autonomous driving, implemented in a physics-driven simulation. We tested our design with various vehicles (from a Tesla Model 3 to an Ambulance) experiencing malfunctioning and swift steering scenarios. We demonstrate significant improvements in dynamic error rate compared with traditional MPC implementation with up to 89.15% median prediction error reduction with 5 spiking neurons and up to 96.08% with 5,000 neurons. Our results may pave the way for novel applications in real-time control and stimulate further studies in the adaptive control realm with spiking neural networks.


Introduction
Over the years, model predictive control (MPC) has steadily emerged as a highly effective control technique widely utilized in various disciplines, with applications spanning robotics [1,2], aerospace [3], and process control [4].This surge in MPC's utility is attributable to its inherent capacity to provide accurate state predictions and subsequent control actions, given a robust and accurate dynamical model.MPC's performance is, however, limited by the model's comprehensibility and accuracy with respect to its realization in real-life systems.Consequently, there is a pertinent challenge that arises in deploying MPC in intricate and stochastic real-world scenarios.Despite going armed with a model that intricately captures the typical behavior of a system, the reality can often be dotted with uncertainties and situations that the model is not designed to handle; ultimately leading to sub-optimal performances or, in extreme cases, triggering system instability due to inadequate control [5].Conceptually, this issue is predominantly rooted in the incongruity between the predictions made by the model and the actual system behaviors that unfold.This discord is further amplified when, as per common practice, the model is assumed to perfectly mirror real system behaviors, translating into a system-model alignment that often fails under real-world conditions.
For instance, in the case of an autonomous vehicle, an inconsistency in acceleration or deceleration performance often stems from irregular load distribution necessitates precise measurements of load dispersion to incorporate into the vehicle model.Further complications may arise with factors such as variation in wheel pressure due to natural wear and tear, extreme circumstances like a tire blowout altering the vehicle's course, or even external factors like changing road conditions like rain or moisture.These scenarios require the development of accurate physical models that reflect individual wheel friction

Model predictive control implementation
The MPC is a model-based control technique that optimizes a control sequence over a finite time horizon.In our implementation, the MPC algorithm uses a state-space model of the system dynamics, which is updated online using continuous measurements from the system.The MPC's objective is to find the optimal control sequence that minimizes a cost function over the prediction horizon.The control sequence is then applied to the system, and the optimization problem is iteratively solved using the updated measurements.An in-depth exploration of the MPC and its various variations can be found within the scholarly work [15].
Our MPC algorithm comprises three main components: the kinematic bicycle model, the cost function, and the optimizer.

The kinematic bicycle model
The kinematic bicycle model (KBM) is widely utilized for modeling vehicle dynamics [16].It represents the fundamental physics of a vehicle's dynamics with a simple and efficient computational model.The primary motivation for selecting KBM here stems from its inherent simplicity and high level of abstraction, eliminating the need and the associated costs of defining a unique physical model for each vehicle type.
KBM approximates the vehicle as a rigid body with two wheels, one at the front and one at the rear, connected by a fixed wheelbase.The model captures the essential dynamics of a vehicle while ignoring certain complexities, such as tire forces and vehicle suspension.A description of KBM, with its advantages and limitations, is detailed in [16].Here we used an extended version of KBM as the base model for the MPC.
The basic KBM is based on four state variables: the position of the vehicle, its heading, its speed, and its steering angle.In our experiment, we used the vehicle's center of mass as the vehicle's position and extended the state variable.Our state of the model at time step t is represented by the vector where (x t , y t ) is the vehicle's position, θ cost and θ sint are the cosine and sine components of the heading angle, respectively, v xt and v yt are the velocity components, a xt and a yt are the acceleration components, and v rt is the angular velocity.
Each control assignment is represented by the vector u t = [ϕ t , δ t ], where ϕ t is the throttle-brake (+1 is considered full throttle and −1 hard brake), and δ t is the steering angle.
To predict the approximate future state s t+1 , given s t and u t , we need to calculate the changing rate (the dynamics d) of each state variable.Here we used the original KBM definitions, which ignore the acceleration and angular velocity terms.To restore the original heading angle, we used: where L is the vehicle wheelbase length, and β is the slip angle defined by β = arctan(L r tan(δ)/L), where L r is the distance between the rear axis to the center of mass.We chose L r = L/2 for simplicity.This introduced disparity further challenged our model to adapt to more demanding conditions, where the model only approximately mirrors the actual vehicle.Note that while the state's heading angle is split into sine and cosine components, we use the original θ angle in the dynamics calculation.The scheme for the KBM with the center of gravity as the reference point is shown in figure 1.
The vehicle's subsequent state is defined by: Although we did not use the extended variables of the state, they will be used in the adaptive model as an additional context as will be described below.

Cost function
We defined a cost function for single state as f : s t × u t × u t−1 → R, where s t is the current state, u t is the current control signal, and u t−1 is the previous control signal (we used u t−1 = [0, 0] for t = 0).
We defined a route to follow as the reference trajectory.The reference trajectory was defined by a series of waypoints with a 1-meter resolution, and 100 km h −1 as a reference velocity.We defined an error vector e = [d e , θ e , v e , ϕ t , δ t , ϕ t − ϕ t−1 , δ t − δ t−1 ], where d e is the cross-track-error (CTE), θ e is the heading-error, v e is the velocity error, and actuators usage penalty.
To calculate the CTE, we build an approximate short continuous trajectory, by taking 60 waypoints starting with the latest waypoint that the vehicle has passed, transferring them from the global coordinate system to the vehicle's coordinate system.Then a 3rd-degree polynomial reference trajectory p(x) was derived from the transformed waypoints.The CTE was calculated as de = p(x t ) − y t .To keep this value normalized we normalized it by 3 meters, d e = de /3.
Similarly, the heading error is calculated by the difference between the heading angle of the vehicle and the reference trajectory θe = θ t − arctan(p ′ (x t )).To keep this value normalized we normalized it by 45 degrees, θ e = θe /45 • .We normalized the CTE and angle to keep each state variable in the range [-1,1], which is the range that the spiking neurons were optimized to represent.
We defined a weight vector w = [550, 200, 1, 110, 250, 0, 50], and defined our state cost function as: Due to the intricate nonlinear behavior of the system, formal methods for parameter selection are generally not readily available or applicable.Therefore, the parameter tuning process often relies on empirical adjustments converging to an adequate model's behavior.Here, the weight vector values were manually tuned to achieve the minimum CTE for the Tesla Model 3 under normal driving conditions with the KBM as the underlying static model.For our MPC, we found that a prediction horizon of N = 5 predictions and prediction steps of ∆t = 0.1 s worked well.Our final cost function summarizes the cost of all previous individual steps: (3)

Optimization solver
The MPC is derived iteratively over a prediction horizon of N prediction steps.The optimization problem, which finds the optimal control signal over the prediction horizon, is solved at each time step, and the process is repeated at the next time step with the new system's state.At each time step the first control input from the optimal solution is applied to the system.The use of the KBM and the optimizer allows us to derive the control inputs that steer the vehicle toward a desired trajectory while respecting the system's constraints.We used the SLSQP optimizer to derive the MPC prediction horizon at each time step.SLSQP is a widely used sequential quadratic programming (SQP) algorithm that can handle both equality and inequality constraints [17].
The optimizer minimizes equation (3) for a prediction horizon of N = 5 (10 variables minimization encompassing throttle-brake and steering for 5 steps).We constrain each variable u i with the constrain −1 ⩽ u i ⩽ 1.

Neural engineering framework (NEF)
The NEF is a theoretical framework, that provides guidelines for designing spiking neural models through the abstraction of individual spiking neurons or ensembles thereof.This framework is anchored in three foundational principles: representation, transformation, and dynamics, which serve as essential building blocks for the development of neuromorphic models [10].NEF has been used to design neural circuits for applications ranging from robotic control [18] to visual processing [19] and perception [20,21].Furthermore, the NEF has been effectively implemented in a plethora of prominent digital neuromorphic hardware architectures such as TrueNorth [22], Loihi [23], NeuroGrid [8], and SpiNNaker [24].Its versatile applicability is also demonstrated in its deployment on dedicated analog circuitry [25].

Underlying principles
In the context of this work, NEF provides the means for the creation, simulation, and analysis of SNNs.NEF's inherent biological plausibility plays a crucial role in crafting SNN models, which we integrated with the MPC algorithm to introduce continuous adaptation.NEF is described in length in [9,10].
A spiking neuron ensemble represents a time-varying vector of real numbers through nonlinear encoding and linear decoding (the representation principle).The encoding process transforms the numerical input vector into neural spikes that represent the input vector.Each spiking neuron in the ensemble has a constant encoder vector e i , a unit length of a vector, which defines the preferred stimulus of the neuron (the value for which the neuron is firing at the highest rate).The activity a of a neuron i for an input vector x is defined by: where G i is a nonlinear function that represents a neuron model (in this work, we use the leaky-integrate-and-fire (LIF) model [26]), α i is the gain term, and J bias i is the bias fixed current.The LIF neuron is defined using: where V(t) is the neuron's membrane voltage potential, J(x) is its input current, R is its membrane resistance, and τ RC is the membrane time constant.The neuron spikes whenever the membrane potential surpasses a threshold voltage, followed by a voltage reset.Decoding the estimated representation of vector x from the neural activity is realized by: where d x i is the linear decoder vector optimized to reconstruct x, h i (t) is a temporal low pass filter, and * refers to mathematical convolution.

The vector d f(x) i
can be optimized to reconstruct any function f (x) using least squared optimization [10] (the transformation principle) by minimizing ⟨(f(x) − f(x)) 2 ⟩ where f(x) is the estimated representation of f (x), calculated by equation ( 6).

Prescribed error sensitivity (PES) learning rule
The PES learning rule is a biologically plausible supervised learning rule, widely used in NEF and neuroscience.It is used to adjust the decoders of neural populations to minimize the difference between the actual output and the desired output (the error).PES is based on the delta learning rule and it only depends on information local to the neuron.The effect of the PES learning rule on the decoders is formulated as where κ is the learning rate, and E is the error vector mapped onto individual neuron.

CARLA and Nengo simulations
In this work, we used the NEF and the Nengo simulation package to develop the SNN model.Nengo is a Python-based software tool that implements the NEF and allows users to build, simulate, and analyze large-scale SNN models [27].
CARLA serves as an open-source simulator that primarily focuses on autonomous driving research [28].Conceived as an environment of high-fidelity virtual realism, it is designed to facilitate the testing and development of algorithms connected to autonomous driving.It encompasses a diversity of pre-integrated sensors and environment-based conditions, incorporating aspects of differing weather and lighting circumstances.
As SNNs are intrinsically time-dependent models, we had to temporally and spatially synchronize CARLA and Nengo simulators.We deployed the CARLA simulator in a synchronous deterministic mode, allowing Nengo exclusive control over the time-space.Consequently, both simulators were coordinated with a physics delta time of 0.001 s to guarantee accurate simulation.

Adaptive MPC controller
While describing an accurate model of system dynamics is hard, it is a critical determinant of the MPC's performance.Here, uniquely, our approach adapts the model to the environment using SNNs.The SNN model is designed to be lightweight and run efficiently on low-power hardware, making it suitable for real-time control applications.The full scheme of the model is shown in figure 2.

Adaptive model predictor
The adaptive model prediction model (encapsulated by an orange square in figure 2) is designed to learn online, using the PES learning rule, the vehicle model in real-time.The MPC (encapsulated by a gray square in figure 2) operates within the same reference time frame as the world.In our specific scenario, it runs in sync with CARLA.
To reduce the error e between the predicted system dynamics d and the actual system dynamics d, we added a parallel spiking neurons population, which can continuously learn non-linear functions online, to minimize the error term: so that d t ≈ dt + êt .
To build that with Nengo, we defined a direct mode ensemble that calculates the KBM dynamics model from equation (1).A direct mode ensemble represents the signals perfectly, rather than through a neural approximation, to reduce noise and minimize the required neural resources (given enough neurons it can be sufficiently approximated).We defined an ensemble of spiking LIF-based neurons, in which its decoder is initialized with zeros to learn the error signal from equation (8).This neural ensemble represents an input vector of 7 dimensions and has a radius of √ 7 (the radius of the sphere used by the ensemble's encoders defines the expected magnitude of the input vector), and it is composed of 5 values representing the vehicle's state (in the vehicle's coordinate system) and 2 values representing the control signal.Both the spiking and the direct ensembles receive as input a state s and an action u.Their outputs are summarized and represent the predicted dynamic d.The error from equation ( 8) is calculated and used as a signal error to the PES learning rule that adjusts the adaptive ensemble's decoders.Our adaptive ensembles get the state s and the action u as a learning context, allowing the network to learn to derive the correct output for a given input.The adaptive ensemble learns to reduce the error e from s and u.The error signal −e t−1 is used to adapt the weights based on the post-synaptic neural activity and filtered it with a lowpass synaptic filter with a synapse time constant of 50 milliseconds.To reflect the real dynamical behavior, we had to delay the predicted output by the prediction time.To do so, we used an ideal delay synapse that delays the predicted dynamics by ∆t.
While we could initialize the model by optimizing the adaptive ensemble decoders to reflect the KBM dynamics from equation (1), PES, as an online greedy learning rule, suffers from catastrophic forgetting [29].While a slow learning rate does not learn quickly enough, a fast learning rate may result in over-fitting to the most recent online example, resulting in losing previously learned knowledge.By defining two dedicated connections, one for the hard-coded model and one for the adaptive learning, we can use a decent learning rate of η = 10 −4 , which can adapt quickly to changes, without forgetting the base model.With higher learning rates, we observed over-corrections and diminished vehicle stability.Conversely, lower learning rates have resulted in slower reactions and responsiveness, which can pose a risk in critical situations.The learning rate needs to be carefully selected to maintain a balance between the speed of adaptation and recovery, ensuring safe driving.This selection should consider the scenarios to be addressed and the extent to which the initial static model accurately represents the actual vehicle.

Adaptive MPC in Nengo
To implement this model, we realized the MPC controller in Nengo and connected it to the CARLA simulator.While MPC optimization is an iterative procedure, that calls the model several times during the same timestamp, SNN models operate continuously.Therefore, we implemented two separate networks that communicate with each other: one learns the system dynamics, and the other uses it for the MPC.We implemented a Nengo interface that connects to the CARLA simulator, which gets an action u t to simulate it, and outputs the new state s t+1 based on the simulated environment (see our GitHub repository for full implementation details).
The state from the environment is transformed from the global coordinate system s to the vehicle's local coordinate system, s L , which rotates the velocity and acceleration vectors such that x = 0; y = 0; θ = 0, thus, we can eliminate them from the state vector and reduce the state vector's dimension.The local state variables s L = [v x , v y , a x , a y , v r ] were normalized by factors [40, 40, 40, 40, 90], respectively, allowing each state variable to stay in the [-1,1] numerical range (the range at which the spiking neurons were optimized to represent).The values were chosen to represent a top speed of 40 m s −1 (144 k h −1 ) and a top acceleration of 40 m s −2 which are above the speed limit we defined for our tests.We calculate the vehicle dynamics using d t−1 = s t − s t−1 , where s t−1 is calculated as an ideal delay connection.The dynamics were continuously compared to the predicted dynamics to derive the error signal using equation (8).
The MPC optimization was implemented as a conventional CPU-based module.It receives the vehicle's current state and provides the optimal control signal.Its output was connected to both the simulation interface, and the adaptive model predictor, for action context.This node executes the SLSQP optimizer to find the optimal control signal in the MPC prediction horizon.While MPC optimization is an iterative procedure, that calls the model several times during the same timestamp, the SNN operates continuously.Therefore, we use another duplicated adaptive model prediction module which operates in a different time reference to be used for the MPC prediction horizon.The model uses a 'frozen' (not adaptable) spiking network.The latest decoders for the adaptive ensembles were copied from the outer network ahead of the optimization.The predicted dynamics were added back to the input state as shown in equation ( 2) to calculate ŝt+1 .Every prediction within the MPC prediction horizon was simulated by the SNN to last 100 ms (which should run faster than the 100 ms in the world time frame).This duration is sufficient for the neurons to reach a state of full spiking activity equilibrium.To simulate an action every ∆t, the node solves the optimization problem every ∆t s.In other timestamps, it returns the last control signal.The connections in both networks have no delay, except the model output which has a synapse time constant of 50 ms.

Experimental design
To ascertain the potency of our adaptive model, a series of experiments were executed.These experiments evaluate the model's performance across different vehicle types and driving conditions.

Vehicle selection
The KBM is grounded on the assumption of non-slip driving conditions and the absence of external forces.Therefore, to rigorously test our model, we have selected a wide variety of vehicle types, each unique in size and kinematics.The vehicles chosen for our experiment include Ford Ambulance, Mini Cooper S, Tesla Cybertruck, Mitsubishi Fusorosa, Tesla Model 3, Ford Mustang, and Volkswagen T2 (2021), as demonstrated in figure 3(E).In equation ( 1), we incorporated the actual wheelbase length, as specified by the vehicle's manufacturer (see the project's repository for full specifications).

Neuronal resources
The number of neurons plays a crucial role in the adaptive model's learning ability-as more neurons are used, the more intricate the learned behaviors can become.However, it necessitates greater computational resources.To critically evaluate the required resources for adequate performance, we evaluated our framework with 2, 5, 10, 50, 100, 500, 1000, and 5000 neurons, allocated for the adaptive ensemble.

Normal driving conditions
In this section, our model operates under standard driving conditions, simulating a 60-second drive on a highway through several waypoints (figure 3(A)).While the KBM inadequately represents the dynamic model of the vehicular types chosen, the MPC learned behavior compensates for it.

Malfunctioning steering conditions
We devised a set of steering mechanism malfunction scenarios, which adds 25% (17.5 • ) of the vehicle's max steering angle (the max steering value for vehicles is 70 • ) while driving on a highway through several waypoints (figure 3(A)), thereby simulating a mechanical failure that can occur in real-world driving situations.

Swift changes recovery
We evaluated our model's robustness against drastic unexpected changes in the steering mechanism.This includes sudden jerks to the right and left every 10 s.We simulated a 40-second drive on a highway through waypoints on a straight path (figure 3(B)).

Assessment metrics
To assess the quality of the vehicle control, we employed three metrics-model prediction error, CTE, and driving smoothness.

Model prediction error
The main factor in evaluating the behavior of the model is the model prediction error, which measures the error signal −e t in each simulation time step as the difference between the predicted and actual dynamics in terms of vehicle positioning, vehicle heading angle, and vehicle velocity.To quantify this metric, we derive the median absolute error over time, making it robust to outliers and expressing a global score of the error.

CTE
CTE gauges the alignment variance between the reference trajectory and the vehicle's position.CTE is not a definitive metric due to its interdependence with the model and cost function.Consequently, it presumes an easier return to the center and wrongly prefers the increase of δ t and decrease d e .A more accurate model, that considers the influence of external forces, would recognize the required greater force to bring the vehicle back to the center and optimize the cost term accordingly.To overcome this issue, a careful fine-tuning of the cost function's weight coefficients for different models is needed.Therefore, to maximize experimental accuracy, we used the CTE for comparing the same type of models.

Driving smoothness
Continuous driving adjustments can lead to irregular steering patterns and error signals.These error signals can potentially disrupt the vehicle's stability and compromise safety.Over-correction, a common result of continuous adjustments, can cause the vehicle to swerve or lose control, particularly at high speeds.Furthermore, it can lead to premature wear and tear of the vehicle's steering system and tires, resulting in increased maintenance costs.Therefore, maintaining steering smoothness is crucial for both safety and vehicle longevity.Ultimately, mastering steering smoothness can lead to a safer, more comfortable, and more efficient driving experience.To objectively evaluate the smoothness of the CTE, we propose the application of short-time Fourier transform (STFT) analysis to the steering data.The STFT, a powerful tool in signal processing, allows us to analyze the frequency components of the steering signals over time.By applying STFT, we could identify the frequency of steering adjustments and detect patterns of over-correction.By analyzing these frequency and amplitude components, we could quantify the smoothness of the steering and identify instances of over-correction.
The first phase involves dividing the CTE into distinct segments, each comprising 1024 samples, resulting in a thorough analysis of the CTE variations.Next, we normalize the segmented CTE based on the average CTE within each segment.This process can be mathematically represented as ĝ(x) = g(x) − g(x), where g(x) denotes the segment's average function.
We realized the fast Fourier transform (FFT) on each segment, deriving the median of all frequencies over time.To encapsulate these analyses into a single quantitative value, we determined the smoothness score by calculating the mean square error of the overall absolute values of the median STFT.This approach provides a comprehensive measure of driving smoothness throughout the sampled data, with a lower score indicating a smoother driving experience.
We sought to establish a baseline for assessing the smoothness of vehicle control.To achieve this, we assembled a group of ten human drivers, comprising four females and six males with driving experience ranging from 7 to 48 years.Each driver examined five experiments involving our adaptive controller under both normal and malfunctioning steering conditions.Examinations were conducted on a full driving scene, from a top-down perspective (as depicted in figures 3(C) and (D)).Following each observation, we asked the driver to binary rate the perceived smoothness of the driving experience (smooth or not).In total, we collected 50 labeled observations.We found that a smoothness score exceeding 1.5 points was agreed by 9 of 10 raters to be unsmooth and induce significant discomfort.

Normal driving conditions
Regarding the vehicle's positioning, heading angle, and velocity prediction, the non-adaptive KBM was demonstrated to have high dynamical error over time with all tested vehicles.We therefore used the adaptive model in both the extended and regular state variables to compensate for it.Our results show that the error was reduced in all scenarios with merely five neurons, reducing the median prediction error by up to 86.14%.The 2-neurons model could not adapt to the changing dynamics and the prediction error was increased over the baseline (table 2).
As decipited in figure 4, in all the vehicles, except for the Mitsubishi Fusorosa bus, the extended state variable succeeded in utilizing the additional acceleration terms in the learning context to better predict the dynamics, even though the additional context is not part of the base model.
The 5000-neuron model was able to adapt accurately to all vehicles.However, the error of the X coordinates for the velocity of the Mitsubishi Fusorosa that came from the basic KBM was not able to be adapted due to the nature of the error.The 5-neurons model was less accurate and stable but was able to reduce the median prediction error by up to 70.22% (table 2).
In our evaluation of driving performance, we quantitatively assessed the CTE and observed considerable variability in the non-adaptive model, despite all tests being conducted on an identical trajectory.As reflected in figure 5, this inconsistency suggests a high degree of variance inherent to the model's operation.
Interestingly, the deployment of adaptive models equipped with a minimum of 5 neurons exhibited stability in CTE performance.As we incrementally introduced more neurons into the system, we observed increasingly similar performance statistics and driving patterns across the board, irrespective of the type of vehicle employed.This suggests that the driving quality is directly governed by the cost function within these models, thus rendering any model errors inconsequential.The adaptive measures effectively mitigate such discrepancies, contributing to a homogeneous degree of performance observed across the diverse set of vehicles.In addition, as we incrementally introduced more neurons into the system, we observed smoother driving, as shown in table 1.With the exception of the Ford Mustang where 10 spiking neurons were required for the adaptive controller to provide smooth driving, for the other vehicles, 5 neurons were found sufficient.The 2-neurons adaptive model showed a dangerous regression in the CTE performance with some vehicle types, causing the vehicle to be unstable.This is expected as we saw a regression in the dynamics error for the 2-neurons model in table 2. Although, in all the experiments the vehicle has never deviated from the lane.

Malfunctioning steering conditions
We evaluated the control performance in a scenario in which the steering mechanism was defective, prompting the vehicle to steer at 25% of the maximum steering angle to the right.As anticipated, the non-adaptive KBM was oblivious to the prediction error, leading to an excessive CTE, and causing the model to veer rightwards.In all trials, this model inevitably collided with an obstacle.Likewise, the model based on a 2-neuron system failed to comprehend vehicle dynamics, as observed in the normal driving condition scenario, yielding subpar performance.Conversely, models that utilized 5 neurons or more had the capacity to adapt to the situation-this included accommodating the dynamic errors and compensating for the vehicle's drift.These models demonstrated similar driving patterns to those observed under normal steering conditions, without marked degradation, as illustrated in figure 7.As the data in figure 6 showed, these adaptive models effectively minimized dynamic errors, echoing the proficiency observed under normal steering conditions.As seen in table 4, the dynamic error measurements from the adaptive model in this experiment closely matched those identified under typical driving conditions in table 2. This reflects the model's capability of reconciling the discrepancies between the static model and actual physics, while simultaneously addressing unexpected steering difficulties.The adaptive model provided up to a considerable 96.08% improvement in median prediction error and up to an 89.14% improvement with just 5 neurons.
Due to the steering issue, vehicles smoothly drifted to the right, where models comprising 5 or more neurons returned the vehicles back to their original course.The slowest response was observed in the Mitsubishi Fusorosa, which had an average recovery time of 4.69 s and a maximum of 5.28 s.The Tesla's Model 3 and Cybertruck demonstrated remarkable recovery times of 1.89 s and 1.915 s, respectively.Upon analyzing the most critical CTE peak caused by the malfunctioning steering mechanism (as shown in figure 7 from 0 to 6 s), we discovered that this was manifested in the case of Tesla's Model 3 and Cybertruck, where this caused a maximum veer of 0.87 m to the right, although did not exceed the lane width (1.75 m).
Post-recovery, we noticed that the driving performance was congruent with our observations under normal conditions across all vehicles, barring the Mitsubishi Fusorosa which displayed erratic driving dynamics with peak values throughout the drive.This was anticipated given the inherent X-coordinate velocity error observed under standard driving conditions, which the steering issue further exacerbated.In conclusion, the smoothness score shown in table 3, though inferior to that of the normal driving conditions in table 1, is low enough to deliver an adequate driving experience despite the mechanical issue.As we increase the number of neurons, the driving experience improves.With merely 5 neurons for the adaptive controller, with the exceptions of the Mitsubishi Fusorosa and Ford Mustang where 50 spiking neurons were required, all vehicles provided a smooth driving experience.While the non-adaptive model and the two-neuron model delivered smooth driving as well, at some point the controlled vehicles veered dangerously beyond the lane boundaries until they collided with an obstacle.

Swift changes recovery
In this scenario, we allowed the vehicle to maintain a straight course, altering the faulty steering direction every ten seconds.As anticipated, the non-adaptive model exhibited a step function-like behavior in CTE, switching its direction from positive to negative according to the steering direction, as illustrated in figure 9.
In contrast, the adaptive model, with five neurons and above, proved capable of compensating for these changes and, in most instances, attained a steady state after five seconds.This success followed the initial overcoming of the previously assimilated adjustment and newly introduced issue.Five neurons or more yielded up to an 84.86% improvement in CTE terms.All the adaptive models, outfitted with five neurons or more, evidenced comparable performance in CTE terms, with an approximate 6 cm disparity existing between the poorest and finest CTEs for each vehicle type.
Despite impressive CTE results and speedy recoveries, we noted that the Tesla Cybertruck, Tesla Model 3, and Ford Mustang exhibited high overshoot post-recovery.Lowering the learning rate might curtail these overshoots, albeit at the expense of lengthening recovery time.
Our observations indicated that models with 50 or more neurons exhibited comparable learning patterns, and the addition of more neurons did not result in a significant reduction in the median prediction error, which remained at an average of 0.03 points.However, superior scores were achieved by models with a neuron count of 500 and above, as shown in figure 8.
Generally, despite abrupt, unforeseen conditions, the adaptive model showcased an ability to swiftly recover and persist in a straight-line motion with comparable quality.

Discussion
Our work introduces a novel solution for real-time control of dynamic systems through the implementation of a continuous adaptive MPC approach that utilizes SNNs.The integration of the SNN model within the MPC algorithm equips the system with the capacity to adapt seamlessly to unexpected environmental variations, subsequently optimizing control performance and enhancing robustness.
Our method's unique feature includes the modification of the MPC model based on the current state and action information.The SNNs, facilitated by NEF and Nengo, can be adjusted and learned online, refining the system's dynamics, increasing prediction accuracy, and overall control performance.This adaptability proves crucial where systems' dynamics change over time or due to unpredictable external factors.
We employed vehicle simulations in our experimental settings, highlighting our approach's effectiveness.When tested in three different experiments and various vehicle types against a traditional MPC controller using a fixed static model, our adaptive MPC controller showcased superior adaptability and control performance during unexpected changes such as steering issues.It was able to maintain control and drive the vehicle, aided by a minimum of 5 spiking neurons.We hypothesize that this is due to the error's low complexity metric and the highly informative nature of the state and action inputs used by the model for output prediction.This allows fewer neurons to effectively capture the underlying patterns.The low complexity of the error could be attributed to the underlying static model, which provides a simple and solid foundation for vehicle control, contributing to the overall efficiency of the model.The base static model might be the reason why fewer neurons are needed to capture the underlying patterns effectively.That being said, we note that smoother driving is highly correlated with an increased number of neurons.We found that a minimum of 50 neurons were required to achieve stable control and ensure a smooth driving experience in all cases.This threshold served as the baseline for the system's performance, balancing computational efficiency with functional effectiveness.A reduction below this threshold may result in a noticeable degradation in control stability and driving experience in some cases.
Furthermore, SNNs provide several advantages over other methods, including traditional artificial neural networks.They are inspired biologically, contributing to efficient, low-power solutions suitable for real-time control applications [9,10].By encoding information through spikes, highly accurate, time-sensitive computations are achievable, benefitting control systems where precision timing is pivotal, particularly in limited power systems like quadcopters.
There are several potential areas for further research.First, the performance of the adaptive SNN MPC controller can be evaluated under more complex and diverse scenarios, thereby assessing its generalizability.Additionally, the examination of different network architectures and training strategies for the SNN model could further optimize its performance.For instance, incorporating legendre memory units [30] layers before the adaptive SNN ensemble might enhance the model's prediction over time by representing the vehicle's movement as the learning context rather than the momentary state.Moreover, the computational requirements and the scalability of the method should be thoroughly analyzed for its applicability in large-scale and limited power control systems, as an increased dynamics complexity necessitates more spiking neurons and increased power consumption.Evaluating the system's performance with other models of spiking neurons might also be an interesting avenue for research.For example, by utilizing spike timing dependent plasticity-driven neurons a biologically inspired learning rule may apply and provide a more efficient learning framework due to its parallel processing nature [31].Finally, this work was evaluated within a physics-driven simulation environment.Our simulation, which is distinctive in its ability to run SNNs as a single entity while managing two distinct time frames, is required for detailed evaluation and validation.It offers the flexibility and control necessary for an initial proof of concept.It enables experimentation with various vehicle types, each with unique physics, across three different scenarios.The simulation results provided us with valuable insights and a proof-of-concept that our model could indeed enhance driving experience and safety.While implementing such diverse and complex tests in the real world would further constrain our results with cost, safety considerations, available neuromorphic hardware, and time, it is an imperative step forward.
In summary, the continuous adaptive MPC approach using spiking neural networks provides an effective control strategy for dynamical systems.The integration of the adaptive SNN model with the MPC algorithm results in enhanced adaptability and control performance.

Figure 1 .
Figure 1.The framework schematic, illustrating the kinematic bicycle model (KBM), one of the road maps used in this study, and the control modules.

Figure 2 .
Figure 2. Control schematic.The adaptive model predictor module (orange rectangle) adapts the vehicle's dynamic, to its simulated physical behavior.While MPC optimization, used for the MPC prediction horizon (black rectangle), is an iterative procedure that calls the adaptive model prediction module several times during the same timestamp, the adaptive model operates continuously.Therefore, we used a duplicated adaptive model prediction module within the MPC, which is a 'frozen' (not adaptable) spiking network.The latest decoders for the adaptive ensembles are copied from the outer network ahead of the optimization.The clock sign indicates an ideal delayed connection.

Figure 3 .
Figure 3. (A) The designated driving route within the environment of Carla's Town 04 map, used for scenarios I & II.(B) The designated driving route within the environment of Carla's Town 06 map, used for scenario III.(C) The Tesla Model 3, with a malfunctioning steering mechanism utilizing a non-adaptive model.Key indicators are highlighted: the shiny blue line marks the ground-truth reference trajectory, the blue line represents the anticipated polynomial trajectory, the green line traces the car's actual position, the red line projects the model prediction from the outer model predictor, and the white points mark the prediction horizon of the MPC of the previous step.(D) The Tesla Model 3, with a malfunctioning steering mechanism utilizing an adaptive model operating with 1000 neurons.The complete occlusion of the red line onto the green implies a high accuracy in model prediction; thus, the vehicle is shown to adhere to the reference trajectory closely.(E) A showcase depiction of all vehicle types utilized within the scenarios.

Figure 4 .
Figure 4.The driving prediction error for the vehicle's positioning, heading angle, and velocity with the non-adaptive kinematic bicycle model, an adaptive model with 5000 neurons, and an adaptive model with the extended learning context with 5000 neurons under normal driving conditions.

Figure 5 .
Figure 5.The CTE with the non-adaptive kinematic bicycle model and the adaptive models for each vehicle under normal driving conditions.The sharp spike at the end of each graph is a lane change where the reference lane changes.The Y axis in each graph represents the lane width about 1.75 meters from each side of the center of the lane.

Figure 6 .
Figure 6.The driving prediction error for the vehicle's positioning, heading angle, and velocity with the non-adaptive kinematic bicycle model and adaptive model with 5000 neurons under malfunctioning steering driving conditions.Driving ceased either after 60 s elapsed or upon the vehicle colliding with an object, whichever event occurred first.

Figure 7 .
Figure 7.The CTE with the non-adaptive kinematic bicycle model and adaptive models for each vehicle under malfunctioning steering.The blue line represents the CTE of the current scenario; the dashed gray line represents the previous scenario result (figure 5).The sharp spike at the end of each graph is a lane change where the reference lane changes.The Y axis in each graph represents the lane width about 1.75 m from each side of the center of the lane.Driving ceased either after 60 s elapsed or upon the vehicle colliding with an object, whichever event occurred first.

Figure 8 .
Figure 8.The driving prediction error for the vehicle's positioning, heading angle, and velocity with the non-adaptive kinematic bicycle model and adaptive model with 5000 neurons under swift malfunctioning steering driving conditions.Vertical gray lines indicate the occurrences of faulty steering.

Figure 9 .
Figure 9.The CTE with the non-adaptive kinematic bicycle model and adaptive models for each vehicle under swift malfunctioning driving conditions.The Y axis in each graph represents the lane width about 1.75 m from each side of the center of the lane.Vertical gray lines indicate the occurrences of faulty steering.

Table 1 .
The smoothness driving score per number of neurons and vehicle type under normal driving conditions.

Table 2 .
The median prediction error per number of neurons and vehicle type under normal driving conditions.

Table 3 .
The smoothness driving score per number of neurons and vehicle type under malfunctioning steering conditions.We marked the best score for 5-neuron or more as the 2-neuron model was not able to complete the driving.

Table 4 .
The median prediction error per number of neurons and vehicle type under malfunctioning steering conditions.