Do I need to sign a Power of Attorney in front of a notary?

Yes. Your signature on the Power of Attorney must be witnessed by a Notary Public.