I personally think it's worth it to have the bigger bags. You can't sell your bags, but you can give them away or use them for storage inside your house (for example, fill a bag with goods and then drop it on the floor).

All items are generally things you can get again, so you can consider donating stuff to a shrine or selling it at auction if you don't want to hang onto it for the time it takes to get a new skill.