Differentiation (derivative) is causal, but not exactly realizable

04/10/2015  

Control engineering, one of the most relevant developments of the 20th century, has many particularities that result somehow strange from the perspective of a computer science student, especially when he/she confronts it for the first time. For instance, it is difficult to assimilate the fact that in a system block diagram all the signals exist at the same time (a computer scientist is told  either to consider sequential or concurrent operations during his/her entire education, rarely simultaneous ones, thus it is common for him/her to consider that each signal is updated only after the subsystem that yields it finishes its internal work).

In this post I am concerned with another such issue: control engineering can become so complex and difficult for the student that, for the sake of devoting enough space in textbooks to explain such complexities, some of the most basic concepts are often just mentioned, without demonstration or even discussion.

Ok, well, that is a general problem with scientific writing that bothers me in a particular manner U_U

Two entangled and shallowly explained concepts in control engineering are whether the derivative of a signal is a causal operation and whether it is realizable. Common answers in the Internet often miss the point, even experienced people find difficult to give a clear and direct answer for this, most textbooks just mention the fact without elaborating it… A hell of an educational nightmare, from my modest point of view.

Here I will try to give an answer from a rather innocent, newcomer (but mathematically educated) perspective. Let’s see if I am able to provide a straight enough explanation… or just contribute with more darkness to the nightmare 😉

1. First of all: when is a system causal?

(Considering a system as something that processes signals to produce new signals; furthermore, considering only SISO continuous-time systems, i.e., those that take a Single continuous-time signal as Input and yield a Single continuous-time signal as Output).

Well, we can define a system as causal iff the signal that it produces is formed just through the use of present and past values from the signal that it receives. Such a system cannot read the future, as it seems logical for physical processes.

From that definition: is it causal a system that derives its input signal? (is the derivative of a function a causal operation?)

Let’s see. The derivative of a function f(t) is formally defined as another function:

\frac{df(t)}{dt} = lim_{h \to 0} { \frac{f(t+h) - f(t)}{h} }

If the derivative exists (is well defined and has some finite value), the limit above exists, which implies that both its left and right limits exist and their values coincide:
\frac{df(t)}{dt} = lim_{h \to 0^{+}} { \frac{f(t+h) - f(t)}{h} } =

= lim_{h \to 0^{-}} { \frac{f(t+h) - f(t)}{h} }

Since we are dealing with real, physical systems and signals, that cannot change their behaviour abruptly in zero time (i.e., that need to change through a sequence of infinitesimal changes), we will assume that, indeed, both limits above exist and coincide, i.e., that the derivative of a continuous-time signal produced by a physical system exists at any time. Mathematically, as it has been kindly pointed out to me by Dr. Luigi Lannelli, we will consider here signals that belong to C^{1}, or, for the sake that they can be differentiated sequentially more than once, to C^{\infty}.

But if the derivative exists, it could be calculated, for instance, with the second limit we have noted previously. Since that limit only uses values of time at present (t) or before (t+h, h \text{ negative}), it must be that, by using the second limit, differentiation is causal. Since the value of the second limit must coincide with the one yielded by the first limit, we must conclude that differentiation is causal for physical signals.

In some places one can read that the derivative is not causal (i.e., it is considered to be uncausal) because it “predicts” the future evolution of the signal, and looking at the future cannot be a causal operation. Oook, I have to admit that I have used that reasoning sometimes (years ago! the offense has prescribed!). But prediction consists of getting the real value of the signal at a future time, and the derivative does not yield that real value: if we know the derivative at time t, we know how the signal is changing at that time, and, from that, we have some knowledge about the signal in the close future… But the exact, real value? Seriously? The best we can do is to approximate the value of the signal at (at least) some specific future time t+h, h \text{ positive}, for example linearly. It is true that the approximation will be better as h gets smaller, but it will never be the real value. The derivative is not predicting the future (it just gives us a hint). We cannot use that reason to establish the causality of differentiation.

2. Now for the second big question: is the derivative realizable?

As before, we need to provide some definition for realizability. In the context of physical systems, realizability is the property of having some way of implementing a mathematically specified system with physical components. So, can differentiation be implemented with physical components? Notice that, due to their physical nature, realizable systems must be causal. What we wonder here is whether the reverse is also true.

The answer for differentiation is no, and although in some places you will read that this happens because the derivative has an unbounded gain at low frequencies (which is true, but also overwhelming if it is read in the first pages of a textbook by a newcomer to Control Engineering), it is due, basically, to the following, much more understandable reason: a physical system cannot provide infinite energy.

Since any input signal, even being bounded in magnitude, can have an arbitrarily large derivative (when the magnitude changes too rapidly), implementing an exact differentiation would force the system to use arbitrarily large amounts of energy. Therefore, it cannot be realizable, at least, in an exact form and for all situations.

Furthermore, the input signal has noise, that is unavoidable in practice. Noise consists of (very informally) unpredictable oscillations superimpossed to the main trend of the signal, with low magnitude but high frequency. The problem here is high frequency and unpredictable: the larger the changes in magnitude due to noise, in a given, short time, the larger the derivative. No matter how small is the magnitude of the noise: if that noise changes rapidly (i.e., its frequency is high), it will have large derivatives. And, unfortunately, we cannot take them into account before operation for all circumstances, because noise, by definition, is unpredictable. Moreover, we cannot get rid of noise (e.g., through filtering) without incurring in other problems, mainly the induction of delays.

At this point, some readers (hello you two!) may complaint: “Hey, wait a minute! I know of some physical system that implements differentiation. Certainly you know. For example:
Opampdifferentiating
In theory, this Operational Amplifier circuit is a system that implements the following transformation of the input voltage:

y(t)=-RC\frac{dx(t)}{dt}

But, again, in the real world things are more complicated than when sketched on paper: that OpAmp needs an external power source to work (typically, \pm15v) that is bounded in the amount of energy it can provide to the circuit, in particular to the output signal y(t). Therefore, if the input signal has high frequency noise or its main trend changes too quickly, the output will be clamped and no longer equal to the derivative. Maybe you consider this to happen only sporadically, but its effects in a real controller can be catastrophic.

I am almost finished. However, since I am a computer scientist and this post is intended (mostly) for computer science students, I cannot leave it here without some words about computational implementations. Even if we try to implement differentiation in a computer, e.g., in an embedded controller, the situation gets no much better: in a CPU the derivative must be approximated by discrete numbers (yes, even when you program in C and are so lucky to have support for floats), which means that we have bounds on the large those numbers can be and on their resolution. For example, we can use the Euler method with some small positive h to implement an approximation to the derivative:

\frac{dx(t)}{dt} \approx \frac{x(t)-x(t-h)}{h}

but notice that, if h is too small in order to have a good approximation, the numerical result can easily overflow the computer number capacities. Worse: we will getting more of the high frequency characteristics of the input signal as we set higher the frequency of sampling (smaller h), making the derivative larger. Still worse! the program must now run fast enough to do all its periodical calculations, including that approximation of the derivative, in less than h units of time, if we want to provide the hard real-time performance needed for controlling a critical system… So yes, we can implement the Euler method in a computer, and make it work ok under suitable trade-offs, but certainly it is not a physical realization of differentiation.

Summary: differentiation is causal for physical signals; differentiation does not predict (exactly) the future; differentiation is not (exactly) realizable; differentiation can be implemented approximately.
facebooktwitterredditlinkedintumblrmail