Generally, yes: a club star party is an event that is open to anyone. But clubs differ, so contact them to be sure. If the event is happening at a public park, it's likely free and open to anyone.
Normally such an event is just as you describe. Members bring their telescopes and let the public look at whatever they're showing. Sometimes these events are more organized, with each scope having a specific set of assigned objects. And sometimes there are handouts or presentations to provide a learning experience.
Check with the club beforehand to be sure.