Five Levels for Differential Equations in Physics
Introduction
Just about any time you want to solve a problem in physics, you’re going to wind up facing a differential equation. In Newtonian mechanics, that means adding up all the forces on an object, plugging that into
—and then solving this differential equation for the position as a function of time.
That’s not too hard for the simplest systems we all meet in our first physics classes, but as you study more and more physics, you’ll very quickly discover that the
That’s because differential equations in general are really hard. But they’re also essential for understanding physics, because they’re by definition the equations that describe change, and physics is all about figuring out how things change with time.
So it’s hugely important to have a toolkit of strategies for tackling the many differential equations you’re going to meet throughout your physics studies. And that’s why you need to learn the five solution methods I’m going to tell you about in this lesson.
We’ll start off with some relatively basic strategies for solving differential equations, which will already take you a long way with lots of problems you’ll meet in classical mechanics and beyond. These include
- Solving by making a substitution
- Using energy conservation
But as we go along, I’m going to introduce you to some more sophisticated techniques, like
- Using a series expansion to solve the equation
- Using an integral transform like the Laplace transform
- Using Hamilton’s equations, which also give us a new way of visualizing the solution as what’s called a flow on phase space.
We’ll see how all these methods work using one of the simplest, but also arguably the most important differential equation in classical mechanics: the equation of a simple harmonic oscillator,
Or, in other words, the
There’s a good chance you’ve run into this equation before, and maybe you’ve already seen a couple of different ways of solving it. But what’s hopefully going to be fun about this lesson is that the five solution methods we’ll discuss will start from the most straightforward, and work our way up through increasingly advanced approaches.
The Equation
First of all, let me quickly remind you where the simple harmonic oscillator differential equation comes from. Our setup is a block of mass
In equilibrium, the spring isn’t stretched or compressed, and the block can sit happily at rest there. Let’s call that position
But if we slide the block away from there, the spring will now exert a force,
Now let’s say we pull the block out to an initial position
The stretched spring pulls the block back toward equilibrium to the left. But then the block overshoots
This is what we call simple harmonic motion. To understand why it plays such an important role in physics, make sure you’ve seen my earlier lesson about it.
To solve this equation, we’re looking for
Method 1: Substituting an Ansatz
It might sound a little silly, but honestly the first thing you can do, especially with a relatively simple looking equation like this one, is to try to guess the solution. Except that “guessing” doesn’t sound very sophisticated, so instead you’ll often see textbooks call it making an “ansatz.”
All that means in this case is, we’re going to ask ourselves if we can think of a function, which, when we take its derivative two times, we get back the same function we started with, times some negative number:
Using our physical intuition that the block is going to oscillate back and forth around equilibrium, functions like sine and cosine might come to mind.
So let’s make an ansatz, and write down a guess of the form
where
We have to have some constants there just to get the units right.
Let’s substitute this guess into the equation and see if it actually works. The derivative of
Now we do it again for the second derivative. This time, the derivative of
This looks promising, because it says that the second derivative of
And that’ll do it! If we choose this value for
So, are we done? Well, no. First of all,
That works because the differential equation is linear, meaning that we only have single powers of
But what are we supposed to do with these two constants
That brings us to a really important point about solving differential equations. The equation itself is only half the story! We also have to specify the initial conditions we want to satisfy in order to get the solution to a problem.
Physically, that makes total sense. When you throw a ball up in the air, we need to know the initial position you’re throwing it from and the initial velocity in order to be able to say what trajectory it will follow. Likewise, we need to know the initial position and velocity of the block in order to say what its position will be after that.
In this case, we released the block from rest at
Mathematically, the fact that we need two initial conditions comes from the fact that the differential equation is second order, meaning that the highest derivative that shows up is the second derivative of
When we plug
Then we had better set
As for the initial velocity,
and therefore we need to set
That leaves us with
where again,
It looks about like we’d expect. The block starts out at rest at the initial displacement
So there we have it, we’ve solved the differential equation—together with the initial conditions—by substituting in a guess with some constants in it, and seeing how to pick the constants in order to get a solution.
This strategy works in general for a linear equation like this:
where
This method is really all you need to solve the simple harmonic oscillator equation—it’s a really simple problem, after all! But as you progress through physics, you’re going to encounter much harder differential equations, and that’s why we’re going to keep going and explore more powerful solution methods in the rest of this lesson.
Method 2: Energy Conservation
As you’ve hopefully learned before, if we take the kinetic energy of the block,
That’s not at all obvious, because both
The way to check that that’s true is to take the derivative of
Let’s check. The derivative of the kinetic energy with respect to
Now pull out the common factor of
The factor in parentheses is just the thing that vanishes when
When we release the block from
Then when we let it go, the block starts to speed up, and the spring starts to relax. By the time it reaches
And we can use this equation for energy conservation to solve for the trajectory of the block. It’s again a differential equation for
Let’s rearrange the equation to isolate
I’ll use the same symbol
Then we can take the square root to get an equation for
Something really special has happened: this equation tells us the velocity of the block as a function of its position
So when the block starts off at
Actually, we should really get minus that because the block is initially moving to the left. So we ought to be a little more careful when we take the square root, since we can get either sign:
We take the minus when the block is moving to the left, and the plus when it turns around and goes back to the right.
And now we can solve for
Next we integrate both sides of this equation:
The integral on the right is super easy: we just get
The integral over
We could also add another integration constant on the left, but we can just absorb that into the other constant
Now we solve for
Cosine doesn’t care if you plug in + or - something—it’s an even function. So we can throw out the
And so we can just set
just like we found with method number one! So conservation of energy also lets us easily get to the solution of our differential equation.
And in fact this strategy can often be successful for harder problems, even when our first method doesn’t work. A great example is the simple pendulum, which is supposed to be so simple that it’s in the name, but actually it’s surprisingly tricky:
The
That’s due to the component of gravity
And yet, this differential equation in general doesn’t have a simple solution for
The reason for the difficulty is the factor of
you can see that it involves infinitely many powers of
So we can’t just substitute in a simple cosine function here anymore and expect to get a solution. (Except in the special case when
With a little geometry, you can write the total energy as
The first term is the kinetic energy written in terms of
If we release the pendulum from rest at an angle
Once again, we can rearrange this equation to get the velocity
Now we separate the variables and integrate:
So far so good. At this point, however, the integral on the LHS is considerably harder than the one we saw before for the harmonic oscillator. It’s called an elliptic integral, and it doesn’t have a simple answer in terms of something like the
Fortunately, though, mathematicians a couple of centuries ago invested a huge amount of effort studying these kinds of integrals, and so a lot is known about their properties. The solution in this case can be written
where the devil is in that “
The result looks like a simple cosine function when
So, this pendulum example shows how energy conservation can give us a different way to make progress with a challenging, non-linear differential equation, even when our first substitution method fails.
But now let’s get back to the harmonic oscillator equation,
So far, we’ve seen two ways of solving it. And these will more or less do the job for most of the equations you’ll meet in your first mechanics class. But if you’re up for it, what I’d like to do now is show you some more powerful methods that will come in handy later on when you’re faced with harder equations.
Method 3: Series Expansion
Using a series expansion is probably the most versatile of all the strategies I’ll show you in this lesson, and you can apply it to most any differential equation to get an exact, or even just an approximate solution.
The idea is, whatever the solution
The question is, how do we figure out what these coefficients are supposed to be?
Let’s start off by imposing our initial conditions. When we plug
And to impose that the initial velocity is zero, we’ll take the derivative of the series:
Now when we plug in
Alright then, so far we’ve figured out
The next thing we need to do is actually plug the expansion into the differential equation,
We’ll need to take the derivative of the series one more time to get the acceleration:
And now we add on
All this needs to vanish if we want our series to solve the differential equation. And the only way that can happen for every time
That second term with a single power of
But notice that there’s also an
We’ve therefore found that
That’s already pretty nice, because it means we get to throw out half the terms in our expansion,
Now on to the even terms. The 0th one says that
and so we can solve for
Next, for the
After plugging in our solution for
We can already see the pattern that’s forming. The first few terms of our series solution are
Does that look familiar? Let’s simplify it a bit by pulling out the common factor of
How about now—does this thing look like the Taylor series for any function you know?
That’s right! The sum in parentheses is just the Taylor series for the cosine! And so, reassuringly, we’ve once again found that
Series expansions like this are an extremely versatile method for solving all kinds of differential equations. You shouldn’t expect that they’ll always sum up to something simple and pretty like this one, though, unless you happen to be looking at a special differential equation with a simple solution. But even if you can’t sum up the infinite series into a familiar function, that doesn’t make it any less useful or valid as a solution to the equation—as long as you’re looking at a point where the series converges.
Method 4: Integral Transforms
Let’s keep it going with our next method: using an integral transform to solve a differential equation.
Now, there are lots of kinds of integral transforms out there—including the Fourier transform, which my last lesson was actually all about—but the one that’s most useful for solving the problem we’re looking at today is called the Laplace transform.
And here’s what it is. The Laplace transform is an instruction to take our position function
What’s left is a function of
Okay… well that sounds like a funny thing to do, especially if you’ve never seen it before. But we’ll see in a moment that this transformation has a magical property when it comes to differential equations.
But the way you should think about it, is that we have two “spaces” here:
To give a couple of examples, if
Or, for our block on a spring, we’ve found—and we’re about to find again—that
Alright, so we can do this integral to go from
The reason is that the Laplace transform acts in a beautifully simple way on derivatives. Look at what happens when we plug
We can simplify this by integrating by parts. In other words, we’ll use the product rule,
in order to rewrite the integrand as
And now we do the integral over
The first term here is just
As for the second term, it’s the integral of a derivative, and so we can write down the result immediately:
When we plug in
And so, we’ve been able to simplify the Laplace transform of
This is the property that makes the Laplace transform so useful for solving differential equations. It says that taking a derivative in
And that means the Laplace transform can turn a differential equation for
Let’s see how that works for our harmonic oscillator equation. We’ll take the Laplace transform of both sides:
Since
On the LHS, we need to use our derivative rule twice in a row in order to simplify the Laplace transform of
And now if we apply the rule a second time to simplify
When we plug in our initial conditions
As promised, there are no more derivatives! The Laplace transform took our differential equation for
And this equation is much easier to solve. Just move the
and then divide out the factor out front to get
And that’s the solution to our problem! In
To finish the job, we just need to transform back to
In fact, I already mentioned that this rational function is the Laplace transform of
And therefore that’s the solution to our original equation.
So that’s method number 4. Starting from a linear differential equation, take the Laplace transform to try to turn it into an algebraic equation, which you can solve for
Method 5: Hamiltonian Flow
Alright, we’re in the home stretch now! And I saved maybe the most fascinating of all for last: Hamilton’s equations and flows on phase space.
We started out with the
Notice that the LHS is the same as
That’s just Newton’s second law: the force is the rate of change of the momentum.
But mathematically, what that enables us to do is replace the single, second-order differential equation that we started with, with a pair of first order equations:
These are called Hamilton’s equations. I haven’t done anything fancy—this pair of equations contains the exact same content as
But working with the first-order equations has a couple of big advantages. To see why it’s helpful, let’s draw a picture with
This picture is called the phase space, and each point in this plane tells us where the block is and what its momentum—or equivalently its velocity—is at any given instant.
So, for example, when we pull the block out to its initial position and then release it from rest, that initial state corresponds to the point shown above on the horizontal axis, where
After we let it go, the block is going to begin to move, and so these
And flow really is a good name for it, because I want you to picture this plane like the surface of a pool of water with some current flowing around it. Then we take something like a ping-pong ball and set it down at the point for our initial conditions. Once we let it go, the current will carry the ball off, moving it around the surface of the water. The flow is the path that the ball follows through the water.
But what determines the shape and strength of the current that’s telling the ball where to move? Our differential equations, of course! We can write the pair of them as a single, vector equation:
Again,
Likewise, we can go to each point
You can see that they’re swirling around the origin—that’s the equilibrium point. And I’m using the colors to indicate how strong the current is: it’s smallest for the yellow arrows near the middle, and gets bigger for the red arrows farther out.
By following those vectors starting from our initial conditions with
This is definitely a more abstract way of thinking about the solution to our differential equation. Remember, the physical system here is the block sliding back and forth on this one dimensional line. So obviously, there isn’t actually any pool of water or ping-pong ball. Those are just useful mathematical constructs for picturing what’s going on.
But what this picture buys us is that we can very quickly understand what the motion of our system is going to look like without solving any differential equations. All we need to do is draw the arrows at each point in phase space that we get from the RHS of Hamilton’s equations, either by hand or better yet on a computer.
That’s already extremely useful. But Hamilton’s equations also give us a direct way of explicitly writing down the solution for a linear equation like the harmonic oscillator, and that’s the last thing I want to show you.
To see that, let’s express the pair of Hamilton’s equations as a matrix equation:
This looks reminiscent of a simple differential equation of the form
which says that when we take the derivative of a function
as we can check:
Our matrix equation for the block on a spring is of essentially the same form, just with vectors and matrices now instead of single numbers:
It says that the derivative of the vector
And the solution is just the matrix analogue of our simple equation for
That way, when we take the
The catch is that
where
That might look like a nasty thing to try to compute, and it certainly can be in general. But for our matrix
Let’s start off by seeing what
That’s convenient!
for
and so on.
Then the Taylor series for
You can see that there are two kinds of terms: the odd powers of
To make the second line look nicer, I’ve also multiplied and divided by
Now some beautifully simple sums are again staring us in the face. On the first line, the sum in parentheses is just
Or, writing out the matrix,
Thus, to get the solution to Hamilton’s equations, we act this matrix on our initial condition vector,
and we find
Lo and behold, we obtain
So I hope I’ve convinced you of how powerful Hamilton’s method is, of converting a second-order equation into a pair of first-order equations. Both for explicitly solving the equation with the matrix exponential, but also for visualizing the behavior of the solution as a flow on phase space.
Conclusion
We’ve now seen how to solve the harmonic oscillator equation with five different, increasingly sophisticated techniques!
Again, nobody’s saying you actually should use Laplace transforms or matrix exponentials to solve such a simple differential equation. But as you work your way up in physics, you’re quickly going to start running into more challenging differential equations where the methods you’ve gotten a glimpse of here become invaluable.
And it’s always a good idea to start learning advanced tools like these by first seeing how to apply them to a simple setup where you already know the solution, before you jump straight into trying to solve much more difficult problems where these tools do become essential.
Let me wrap up with a few last comments. First, this was hardly a complete list. There are many different strategies for solving differential equations that I didn’t cover here, and the best one to use depends on the equation you’re trying to solve. For example, one of the most ubiquitous in physics is the method of Green functions, which is also one of my favorites. But that really deserves a dedicated lesson.
Second, a really useful approach for solving hard differential equations is to try to tackle them in a special limit where they become simpler. I talked more about that in my lesson about Taylor series.
And third, remember that it’s not cheating to use a computer to help you solve a differential equation. Either by finding an exact solution, or a numerical approximation to the solution. There are lots of different tools for doing that, but one place to start is just to type your equation into wolframalpha.com.
See also:
If you encounter any errors on this page, please let me know at feedback@PhysicsWithElliot.com.