Yeah, I’m lucky if I can get below 45 lbs with all equipment and water on my backpack hunts but that’s usually a min of 5 days of food. I would say my avg pack-in time is anywhere from 3-5 hours in the units I hunt (mainly AZ and NM). From that point I may or may not hunt from a base camp, depending on the usual factors.I'm very curious how one goes backpack hunting without actually carrying a backpack. An example. Lets say you go on a 5 day hunt, you only walk around in the daylight, and daylight is 12 hours per day. So that's 60 hours of hikeable time. 0.01% x 60 hours = 36 minutes...and that's round trip, so 18 minutes each way. This would imply that your camp is at most about 1 mile from the truck. Also this would imply you didn't kill anything that you had to carry out, which is reasonable since you're 18 whole minutes from the truck.
A reasonable daypack with weapon, optics, clothes, and food is 20 pounds. 20 pounds on your back at 11,000ft is definitely noticeable. Most backpack hunters never go anywhere without their day pack.
I do agree that carrying heavy weight all year will just beat your knees and shins to pieces. Staying in shape year round helps a lot, but that can just be staying strong and not fat. Crushing cardio in February for a September hunt is totally useless. Unless you're a big ol fat guy that needs to cut a bunch of weight, just lift weights year round then ramp up the cardio 4-6 weeks prior to a hunt for conditioning. It takes far less time to build cardiovascular fitness than physical strength and muscle.
Above all, mental toughness will get you farther than physical fitness. However pushing yourself with physical training also builds mental toughness.