Can I change my default invitor?

NO. The referral relationship is permanent. 

But by using an explicit invite link will temporarily overwrites the relationship.