While aquaculture is a rapidly growing sector of the U.S. food industry, many wild capture fisheries are contending with financial, social, or ecological challenges that threaten fishers’ abilities to maintain their livelihoods, leading to suggestions that fishers could diversify or transition into aquaculture. However, research looking at commercial fishing to seafood farming livelihood diversification or transitions in the U.S. is limited. We examined how commercial fishers and shellfish farmers in Maine, New York and Florida perceive the shift (diversification or transition) from commercial fishing to shellfish aquaculture as part of their livelihood strategy. Using data we collected from interviews with commercial fishers, shellfish farmers and fisher-farmers, we found that fishers are perceived as having valuable skills on the water and experience in the seafood industry that non-fishers often lack and could make them well-suited to seafood farming. Alternatively, non-fishers are equipped with other valuable skills and experience that fishers often lack, such as business management, marketing, site mapping and design, or even comfort staying in one spot for long periods of time. Overall, transitioning or diversifying from commercial fishing to shellfish farming was not seen as a more logical alternative livelihood strategy for fishers than for non-fishing professionals. The suitability of aquaculture for fishers was perceived to be most influenced by sociocultural factors and psychological attributes. As aquaculture investment and expansion in the U.S. continue to increase, our research suggests that policies or programs intended to facilitate livelihood diversification or transitions into shellfish aquaculture should consider targeting groups beyond just fishers, and account for the ways that local social and cultural factors influence seafood-based-livelihood strategies.