I had a 10 year career in higher ed before my first job in tech, and I'm very thankful that I made the jump. I know everyone's circumstances and desires are different - my upbringing was actually the opposite of what you described (take your time, its okay to wander a bit) - but now I'm trying hard to make up for "lost time" and tech is definitely helping me do that.