The SSO extension type creates a SSO linkout that works with Q2 platform features. It is more limited than a standard Tecton extension but it can support the following types of popups:
- Overpanel (Iframe floating over the page)
- New tab
Based on your question, it sounds like you will want to implement a Tecton extension and peform openUrl functions (see here) and implement other features such as modals and supporting incoming parameters in the self.form_fields object.