The hour is not the same though.
The hour is identical. What is done and how we feel about that hour is different.
The hour in front of the TV is resting and recovering while the hour at the gym has a bunch of logistics, requires work to figure out an exercise plan, and then requires additional rest to recover from the extra effort.
Rest and recovery is largely an opinion too. For instance, smokers say smoking relaxes them - is it true? No. What it does is suppresses a craving for a little while. What we choose to do matters, and how we feel about it matters less in some cases, given what is being done.
I suspect that too much TV is like smoking. It doesn't actually relax us, but makes us crave more relaxation. So we keep feeding it, upping our daily intake. How much downtime do people need? 10% 50% ?
I also suspect that the recovery from watching TV too much is harder than going to the gym to maintain health. Like many excuses, the hidden cost of using them is higher than the cost of doing what we said we couldn't.