I grew up fishing aroud the islands in the Gulf of Maine. I watched flounder and cod all but disappear. Halibut followed the decrease in size you describe. I saw the rise and quick fall of fisheries that targeted spawning times and behavior such as Maine shrimp and sea urchins. Others that used indiscriminate practices such as trawlers for groundfish and scallops decimated the bottom as well as taking all species with their drags. Add to that invasive species that replace productive habitats with monoculture type growth and there we were, living on a island with no fish to be had except the salmon in the aquaculture pens. Aquaculture and sea-farmng may be seen as a success but it is a very different vision indeed.