A trip out to nearby Coconut Island along the footbridge is a good idea. From there you can watch local kids swimming in the water of the bay or diving off a constructed platform. There's even an enclosed area there safe for children to swim in. The Hawaiian name for the island is actually Mokuola ("healing island"), and, according to legend, a dunk in the water around the island will cure what ails you, assuming it's physical. There are also tables on the island for picnicking. 

Banyan Tree Drive, along which the gardens are located, is also well worth a stroll. Park your car on one end of the drive and walk the half mile (one way) down the banyan bordered street. Viewing the sheer girth of some of these trees you might be amazed to know the oldest are less than a century old. Hilo's climate accelerates things. At places the trees block out the sun in a tunnel of green. Look closely at the plaques beneath some of the trees to see who planted them. You'll find Babe Ruth, Richard Nixon, and Amelia Earhart among other names. 

Hilo is a very safe city, for its size probably the safest in Hawaii, but it's not recommended you go walking the gardens late night, even though you will find them open.