Now that Internet access is so common, you can do things more spontaneously, but it is still very important to prepare appropriately for your library visits. This means checking that the library in question has the material that you want to use, that the library is indeed open (they close regularly for inventory and cleaning) and that you are allowed to use it. You may need a letter of introduction from your professor or librarian, for example. Be sure to check the library's home page for more information.
The National Diet Library is open to anyone over the age of 20, but since the materials are almost all in closed stacks, you need to be prepared to make requests, wait for service, wait for photocopying, and there are limits on how many materials you can request at one time. There are things that only the NDL has though, and service is much much better than it used to be. It is now possible to put items on hold before you go to help speed up the process.
Prefectural libraries or large municipal libraries often have wonderful collections, particularly if you are working on modern literature or Japanese history, and much of their collection is in open stacks so you can browse and read to your heart's content. Don't hesitate to visit the reference desk and ask for help. The librarians that I have spoken to have been very kind and helpful.
If you are more interested in reading for fun, check out your nearest public library. They are often filled with students studying for exams, older people reading, and kids running around. You will find current magazines, lots of novels, and a good selection of kids' books.
Private university libraries - like Waseda - require letters of introduction. Waseda, in particular, has one of the best Japanese literature collections, and is the favorite of foreign scholars. So they are strict about following protocol. Please be sure to follow the rules and prepare accordingly.
National university libraries used to require letters of introduction but do not any longer. You will need to register as a guest to be admitted into the library. Branch libraries - at the University of Tokyo for example, may have different rules. Again, you should read their web pages very carefully.
