It sounds to me like you appreciate learning about science, rather than doing science. There's nothing wrong with that.
There are courses in science for liberal arts majors whose labs are less math-intensive than those for science majors.
The typical road to a degree in astronomy is via physics, since more universities offer that major than offer a degree in astronomy per se.
However, there are other routes.
If you're interested in history, then perhaps a liberal arts degree in history with a minor or concentration in various science courses would be more to your liking.
Perhaps you could major in education and pursue a broader exposure to science in general, rather than focusing on one area, and then teach at the public-school level or work in space science at a museum or other public institution.
Or, if you enjoy writing, you could work toward a degree in creative writing or literature and specialize in science writing -- or even technical writing.
You can actually DO astronomy (and many other sciences) as an amateur without becoming so proficient in the math and physics that could achieve an advanced degree in the topic. The word amateur, for example, means "one who loves" rather than "one who is not proficient". I have known several people whose fondness for astronomy has diminished after rigorous coursework in pursuit of a degree.
You will need enough math and physics to understand the coursework, and to be able to explain it satisfactorily, but that doesn't mean you need to become a practicing scientist.
Does that help?