Illustration on real winds (ERA5 reanalysis, September 2017). The balloon only changes altitude; each altitude has its own wind. (The route shown was computed with the wind known to show the setting; it is not a result from the paper, in fact it's less impressive!)
A weather balloon π cannot steer itself. It can only go up or down βοΈ, and the wind π¬οΈ at each height carries it somewhere different. To reach a target π― it has to ride the tide π, but it doesn't know where the flow is going! (well maybe it pays attention to weather forecasts π‘ before going out): it only knows the wind where it has already been.
So it has two jobs at once: learn the wind, and get to where it is going. Should it take a detour to learn more (explore π§)? or head for the target on a shaky guess (exploit π²)?
Illustration on real winds (ERA5 reanalysis, September 2017). The balloon only changes altitude; each altitude has its own wind. (The route shown was computed with the wind known to show the setting; it is not a result from the paper, in fact it's less impressive!)
This is ongoing work on making decisions in settings with exploration-exploitation tradeoffs, when you donβt know the underlying dynamics and constraints, and can only get partial observations. More soon.
The simulation environment behind this work is zermelo.