1. In '/admin/structure/webform/config' check "Enable site-wide dialog support".
1. In '/admin/structure/webform/manage/YOUR_FORM/settings' you can test in Dialog Settings section.
1. In your code on your button/a href add class you can find in the Dialog Settings section.