While I wouldn’t necessarily recommend it, a stranger off the street could sneak into almost every single non-seminar class I’ve taken at Harvard, for several reasons:
(1) Easy access
Most classes are held in large buildings like Sever, Emerson, Harvard Hall, and the Science Center, none of which have any sort of swipe access restrictions.
So, you wouldn’t even need to wait for someone to hold the door open for you in order to sit in on Harvard’s most popular humanities/engineering courses.
(2) No seating charts
Not a single class I’ve taken at Harvard has a seating chart. Not even seminar-style courses with 12 students around a table. Not even the graduate courses I’ve taken. Seating charts are very “high school”-y and teachers don’t waste time on them.
Now, most students end up sitting in the same seat every day, so there typically becomes a de facto seating chart, but if you sat in someone’s seat they likely wouldn’t bat an eye and would just sit somewhere else (assuming there were enough seats for everyone)
Klarman Hall at HBS, where one of the more popular General Education courses about tech ethics was held this year ( Image Source )
(3) Large lectures (mostly STEM courses)
Most engineering courses at Harvard take place in large lecture halls, so you could easily slip in without anyone noticing.
In my experience, only upper-level and graduate-level seminars would be too small for you to enter a classroom without getting noticed.
(4) Extension/visiting students
Even if you didn’t look like a Harvard student at all, you still wouldn’t stick out.
That’s because there are plenty of older Extension students (i.e. actual adults) who sit in Harvard courses, visiting students from MIT, and one-semester junior transfers from abroad.
Students in the Harvard Science Center take a Math 21b exam (Image Source)
(5) Non-mandatory lectures
Some of the largest lecture courses are recorded. This means that students can skip lecture and watch them online after the fact.
Additionally, some courses do not require attendance.
This means that not even the professor will know who truly is in his/her class, and neither will the students.
To summarize, your main difficulty will be figuring out where/when classes are being taken, not getting in.