A central notion in dynamical system theory is that of a "state- space" that separates between the (past) input and the (future) output. The state- space dynamics captures all the past information required in order to propagate the system. Interestingly, this structure is somewhat analogous to joint source and channel coding in Shannon's classical model of communication. Joint source-channel coding amounts to finding efficient representation of the source (past) structure that enables its sufficient (future) reconstruction - within possible distortion - across a noisy channel. We argue that this analogy can be made formal, via the information bottleneck framework, and completely explicit for the case of linear time-invariant systems with Gaussian innovation and state noise. For that case the system transfer function can be revealed from the input-output statistics alone, through a cascade of phase transitions, by trading between the input- state and state-output mutual-information. Those transitions reveal sequentially the pole-zero structure of the underlying linear dynamical systems.

Based on a joint work with Felix Crutzig and Amir Globerson.