Modeling and control of intelligent robots
from standard methods
Gastone Pietro Rosati Papini
http://tonegas.it - gastone.rosatipapini@unitn.it
University
Bachelor's Degree
Software Engineering
- Programming languages
- Software skill
- Realtime systems
- Networking devices
Master's Degree
Automation engineering
- Mechatronics system
- Mechanic modelling
- Control theory
- Robotics
:(){ :|:& };:
Control System - on single joint
Results - Testing of the system
Model based approach - Explicit self tuning control
The control system is an adaptive controller defined as:
- explicit self tuning control
The regulator is updated through the system parameters.
In our case the characteristics are:
- The controller is model based
- Parameter estimator (estimation) is model based
— Åström, Karl J. and Wittenmark, B. Adaptive control. Courier Corporation, 2013.
PhD - Main Activities
- Veritas (FP7-ICT 247765) - European project focused on empathic design
A desktop haptic device is employed to induce a programmable hand-tremor on healthy subjects
Veritas Project - Desktop haptic interface for hand tremor induction
Parkinsonian user
— Rosati Papini, G.P. et al."Haptic hand-tremor simulation for enhancing empathy with disabled users." 2013 IEEE RO-MAN.
— Rosati Papini, G.P. et al."Desktop haptic interface for simulation of hand-tremor." IEEE Transactions on Haptics, 2015.
Human impedance estimator
Why is needed?
- Because the system is used by different person
- RMS amplitude with time window of 1 sec: \(E(s)\)
- Filtered wrist reference position: \(x_{pw}^*\)
- Filtered wrist position: \(\widehat{x}_{uw}^*\)
- Human impedance estimator is a PID regulator: \(M_e(s)\)
- Estimated human impedance: \(\hat{m}_{h}\)
System compensator
Position estimator
Results - Testing of the device
Designer of Indesit company testing our device on a gas hob
— Rosati Papini, G.P. et al."Desktop haptic interface for simulation of hand-tremor." IEEE Transactions on Haptics, 2015.
Model based approach - Model reference adaptive control
The control system is an adaptive controller defined as:
- Model reference adaptive systems
The adjustment mechanism set the controller parameters in such a way that the error between \(y\) and \(y_m\) is small.
In our case the characteristics are:
- the controller is model based
- reference amplitude estimator (model reference) is not based on physical model
- mass estimator (adjustment mechanism) is not based on physical model
— Åström, Karl J. and Wittenmark, B. Adaptive control. Courier Corporation, 2013.
PolyWec Project - Exploiting electroactive polymers for wave energy conversion
- Preliminary studies on the energy production of Poly-OWC
Compiled simulink schema for the energy production evaluation - Hardware in the loop tests
Testing different control schemas and harvesting cycles - Experimental tests
Implementation of control schema and video analisys for energy harvesting evaluation - Optimal control for Poly-OWC
Real-time controller for maximizing the energy estration of the Poly-OWC
Control of an OWC wave energy converter based on DEG
Results - Optimal Cycles
— Rosati Papini, G.P. et al. "Control of an OWC wave energy converter based on DEG." Nonlinear Dynamics, 2018.
From optimal control to a real-time controller for Poly-OWC
- From direct inspection of the results of the optimal control is obtained real-time controller
- Lookup table defines the optimal time to charge using \(\dot{p}_{th}\) based on \(H_s,T_e\)
Model based approach - Gain scheduling controller
The control system is an adaptive controller defined as:
- gain scheduling controller
the gain schedule adjusts the controller parameters on the basis of the operating conditions.
In out case the characteristics are:
- the control logic is derived from optimal control solutions
- the lookup table (gain schedule) is based on incoming wave parameters
— Åström, Karl J. and Wittenmark, B. Adaptive control. Courier Corporation, 2013.
Dreams4Cars (H2020 731593)- bio-inspired artificial autonomous agent
The cognitive architecture is applied to autonomous driving
- The agent use the motor cortex concept, that is a control space
- Each point in the motor cortex in an action and encodes a minimum jerk trajectory
- The points are weighted using the action biasing
- The best trajectory is choosen using action selection
- Then it is converted to control signal by inverse models
— Da Lio, M., Riccardo, D. and Rosati Papini, G.P. "Agent Architecture for Adaptive Behaviours in Autonomous Driving" working progress...
Dreams4Cars - Activities
Each activity covers a different branch of the cognitive architecture
-
Stability and robustness analisys of vehicle lateral control based on dynamics quasi-cancellation:
- Action priming → Action selection → Motor output
-
Robust decision making based on multi-hypothesis sequential probability ratio test:
- Action selection
\( \begin{array}{l} \hline \text { MSPRT algorithm } \\ \hline \text{ Result: Action log-likelihood } \\ \mathcal{M}_{\text {list }} \leftarrow \mathcal{M}_{t} ; (\text{with}~\mathcal{M}~\text{motor cortex}) \\ \overline{\mathcal{M}} \leftarrow \text{mean}\left(\mathcal{M}_{\text{list}}\right);\\ \text{compute likelihood: } L(t) = \overline{\mathcal{M}}-\log \sum_{k=1}^{N} \exp \left(\overline{\mathcal{M}}_{k}\right) \\ \text { if } \max (\exp (L))>\text { threshold then } \\ \quad \left| \begin{array}{l}\text{ take}~\text{action}~\text{with}~\text{higher}~\text{evidence};\\~\mathcal{M}_{\text {list }}=\lambda \overline{\mathcal{M}}; \end{array} \right. \\ \text { else } \\ \quad \left| \text { follow previous action; } \right. \\ \text { end } \end{array} \) -
Flexible modelling of vehicle dynamics with neural networks:
- Simulation and Motor outuput
-
Deal with uncertainties via reinforcement lerning:
- Simulation for Action biasing
Modelling longitudinal vehicle dynamics with neural networks
Causal system
\( \definecolor{myRed}{RGB}{234,26,8} \definecolor{myGreen}{RGB}{50,153,5} \definecolor{myBlue}{RGB}{0,10,206} \definecolor{myYellow}{RGB}{205,255,3} \begin{aligned} {\color{myYellow}a} &= \frac{1}{M} \sum_{i=1}^{m} F_{i} \\ &= {\color{myRed}F_\text{b}}+{\color{myGreen}F_e
g_r} + {\color{myBlue}F_a} + {\color{white}F_s} \end{aligned}\)
Network models for longitudinal vehicle dynamics
Neural networks with inspired physical structure
Convolutional
Recurrent
Unstructured networks
Convolutional
Recurrent
Network Predictions - time series analysis
— Da Lio, M., Bortoluzzi, D. and Rosati Papini, G.P. "Modelling longitudinal vehicle dynamics with neural networks." Vehicle System Dynamics, 2019.
Physical inspiration for data driven approach
The model of the system is realized by neural network that is a:
- differentiable computational graph
- gradient descent algorithms are applicable
In this framework is simple to learn parameters of the model
In our case the characteristics are:
- this approach is flexible and modular
- the structure of the neural network takes inspiration from equations of motion
- the network is no black-box but it is explainable
- less parameters, less data for training, no overfitting, less experiments
— Goodfellow I., Bengio Y. and Courville A. Deep learning. 2015.
Use neural network for control
Synthesis of neural network for inverse modelling
using direct and inverse neural network in series (unsupervised method)
Neural Network for RL
The neural network chooses the requested cruising speed:
- the network possible actions are \(a=\{-\Delta_\text{RCS},0,\Delta_\text{RCS}\}\)
- the network estimates future reward for each possible actions:
- \(Q(s,a)\approx E\left[\sum_k^{\infty} \gamma^{k} r_{t+k}| s, a\right]\)
- a positive reward is given when the car reach without impact the end of the road
- during the simulation data are collected to train the network to a better estimation
Results - CRF Test of transfer learning
Explainable RL approach on a focused issue
Deep Q-learning a reinforcement learning method
- temporal difference
- off policy
- value-based
- model-free
In our case the characteristics are:
- the network used is simple and therefore explainable
- use reinforcement learning only to learn what it is needed
- the network is integrated with an autonomous agent
- the transfer learning is applicable
- it is possible to synthesize a simple logic from the NN behavior
Robots - Quadruped
Collaboration: Andrea Del Prete researcher Dep. Industrial Engineering UniTN
HyQ Blue iit robot granted to DII UniTn
Challenges/Opportunities:
- Estimation of perturbation forces
- Estimation of contact forces
- Handling structured objects (doors, drawers, etc..)
- RL application high-level behaviours
- Extension of the notion of motor cortex as way for motor control planning
Vehicles - Driving simulator
Collaboration: Prof. Mauro Da Lio Dep. Industrial Engineering UniTN
Challenges/Opportunities:
- Integrate the autonomous agent with the simulator
- Use autonomous agent for realistic traffic
- Studing the human robot interaction, where the human influeces the behavior of the autonomous agent
DII driving simulator
Vehicles - Real vehicles
Collaboration: Prof. Paolo Bosetti Dep. Industrial Engineering UniTN
DII Formula SAE Project
Challenges/Opportunities:
- Integrate the autonomous angent with the driving system of the car
- Develop a inverse network for driving to the limits
- Test robust network in real application
Smart materials - Solve modelling
Collaboration: Prof. Marco Fontana Dep. Industrial Engineering UniTN
Use physical inspiration to build a neural network model of electroactive polymers
\(f_\text{tot} = f_\text{e}(\lambda)+ f_\text{d}\left(\left[\lambda_t,\ldots,\lambda\right],\left[\dot\lambda_t,\ldots,\dot\lambda\right]\right) + f_\text{v}(V,\lambda)\)
- total force realized: \(f_\text{tot}\)
- streach: \(\lambda\)
- voltage: \(V\)
- elastic force: \(f_\text{e}\)
- viscoelastic and isteretic force: \(f_\text{d}\)
- electric force: \(f_\text{v}\)
Challenges/Opportunities:
- Estimation of streach-force response over time
- Estimation of force-streach response
- Optimization of the test experiments
Other works - Predictions of human motion by structured neural network
Collaboration: Prof. Daniele Fontanelli Dep. Information Eng. and Computer Science UniTn
The network is based on social force model
- \(\mathbf{f}(t)=m \frac{v^{d}(t) \mathbf{e}^{d}(t)-\mathbf{v}(t)}{\tau}+\sum_{w} \mathbf{f}_{w}^{W}(t)\)
- forces on pedestrian: \(\mathbf{f}\)
- pedestrian mass: \(m\)
- pedestrian speed: \(\mathbf{v}\)
- desired velocity: \(v^{d}\)
- waypoint direction: \(\mathbf{e}^{d}\)
- velocity chage rate: \(\tau\)
- obstacle forces: \(\mathbf{f}_{w}^{W}\)
Challenges/Opportunities:
- pass to headed social force model
- estimation of more waypoints
- include pedestrian interaction
— Antonucci, A., Rosati Papini, G.P., Palopoli, L., Fontanelli D. "Generating Reliable and Efficient Predictions of Human Motion: A Promising Encounter between Physics and Neural Networks" 2020 IROS, submitted.