openpyxl的alignment函数中的参数：wrapText=True，就可以了

from openpyxl.styles import Alignment
worksheet.cell(row=1, column=1, value=value).alignment = Alignment(wrapText=True)