No, you don't have to be a swimmer.  The flotation belt will keep you afloat, however, we do recommend that people be comfortable in the deep water as the flotation belt is not designed as a life saving device.