Tamsui is at the end of Taipei's Red Line. The Old Street has a variety of restaurants and shops and you can reach the oceanfront from here, in contrast to the bulk of Taipei which is several miles inland.
The Chiang Kai-Shek Memorial Hall is one of Taipei's most distinctive landmarks. The museum on the bottom levels paints an over-the-top picture of Chiang Kai-Shek's life. The 228 Memorial Hall a few blocks away is worth a look for a more balanced take on the excesses of the Kuomintang.