Penetration Testing

FREQUENTLY ASKED QUESTIONS

Questions & Answers for questions we commonly receive.

Penetration Testing

What is a penetration test and how is it different from a vulnerability scan?

A vulnerability scan uses automated tools to identify known weaknesses — it tells you what software versions are running and which CVEs apply. A penetration test is conducted by a human practitioner who thinks through attack chains, validates exploitability, and attempts to demonstrate what an adversary could actually accomplish in your environment. The scan tells you what might be vulnerable. The penetration test tells you what is actually exploitable — and what the real-world impact would be.

What is the difference between black box, gray box, and white box testing?

Black box testing simulates an outside attacker with no prior knowledge of your environment. Gray box provides partial information — typically credentials or network diagrams — simulating an attacker with limited inside knowledge. White box provides full access to documentation, source code, and configurations, enabling the most thorough assessment. Most engagements use a gray box approach, which balances realism with efficiency.

Will the penetration test disrupt our operations?

Disruption is rare but not impossible, and we take every precaution to prevent it. We agree on rules of engagement before any testing begins — including testing windows, out-of-scope systems, and emergency stop procedures. You will always have a direct line to the practitioner conducting your test throughout the engagement.

What is a penetration test and how is it different from a vulnerability scan?

What is the difference between black box, gray box, and white box testing?

Will the penetration test disrupt our operations?

Does our organization actually need a penetration test?

Penetration testing is now an explicit requirement baked into many U.S. regulations and compliance frameworks — most commonly in regulated industries like defense contracting, financial services, critical infrastructure, and healthcare. FFIEC requires it for financial institutions. Proposed changes to the HIPAA Security Rule would for the first time mandate annual penetration testing for all covered entities and business associates handling electronic protected health information. CMMC requires it for DoD contractors. If you are subject to any of these frameworks, the answer is yes.

Will a penetration test satisfy our NCUA or FFIEC examination requirements?

Yes — when scoped and documented correctly. FFIEC examiners expect to see evidence of periodic penetration testing as part of your information security program. Thorium's reports are written with examiner review in mind and include an attestation letter confirming the engagement was conducted.

How often should we conduct a penetration test?

Annual testing is the baseline requirement for most regulatory frameworks including FFIEC, HIPAA, and CMMC. Organizations that undergo significant infrastructure changes — new systems, cloud migrations, acquisitions, major application updates — should consider additional testing following those changes regardless of the annual cycle.

Can we use the penetration test report for our cyber insurance application?

Yes. Many cyber insurance underwriters now require evidence of annual penetration testing as a condition of coverage or favorable rates. Thorium's reports include the documentation and attestation language most underwriters look for.

How long does a penetration test take?

Duration depends on scope. An external network test for a small organization typically takes two to five business days. Internal network tests, web application assessments, and larger environments take longer. We provide a specific timeline estimate during scoping and stick to it.

Do you test third-party systems or cloud-hosted applications?

Yes, with appropriate authorization from the relevant service providers. Many cloud platforms — AWS, Azure, Microsoft 365 — allow penetration testing under specific terms. We confirm authorization requirements during scoping and help you navigate the approval process with your vendors.

Who actually conducts the penetration test?

Senior practitioners — every time. Thorium does not use junior analysts or hand off work after the scoping call. The practitioner who discusses your engagement is the practitioner who conducts it. This is not common in the industry and we consider it non-negotiable.

How is Thorium different from other penetration testing firms?

Our practitioners came from U.S. government offensive security environments where the work had to be right. We brought that standard to the private sector. We work with a limited number of clients at any time, which means your engagement gets our full attention. We have no product to sell and no vendor relationships that influence our findings. Every recommendation we make is based solely on what is right for your organization.

Do you sign NDAs and business associate agreements?

Yes to both. We sign NDAs as a standard part of every engagement. For healthcare clients subject to HIPAA, we execute a Business Associate Agreement before any work begins.

Regulatory Compliance

What is the difference between a compliance audit and a risk assessment?

A compliance audit measures your control environment against the specific requirements of a regulatory framework — FFIEC, HIPAA, PCI-DSS, CMMC — and identifies where you do not meet those requirements. A risk assessment identifies threats, evaluates the likelihood and impact of those threats, and quantifies your overall exposure. Most frameworks require both, and the two engagements complement each other. Thorium conducts them as separate, standalone deliverables or as a combined program.

We have an upcoming NCUA examination. How quickly can you help us prepare?

We can begin a scoping conversation within one business day of your inquiry. Examination preparation timelines depend on how many gaps exist in your current program — organizations with documented policies and recent assessments can move quickly, while organizations starting from scratch require more lead time. Contact us as early as possible. Rushing compliance documentation in the weeks before an examination is one of the most common — and preventable — reasons organizations receive findings.

Does passing a compliance audit mean we are secure?

Not necessarily — and this is a distinction we take seriously. Compliance means your documented controls meet a framework's requirements at the time of assessment. Security means those controls are actually working, consistently applied, and protecting your organization against current threats. The two overlap significantly but are not identical. Thorium conducts compliance audits as substantive technical and procedural assessments rather than documentation reviews — which means the gap between compliance and actual security is smaller at the end of our engagements than it is for organizations that treat compliance as a checkbox exercise.

Can you help with multiple frameworks at once?

Yes. Many of our clients are subject to multiple overlapping frameworks — a credit union may need to address FFIEC, GLBA, and NIST simultaneously. We assess controls once and map findings across all applicable frameworks, which reduces duplication and makes the most efficient use of your time and ours.

What happens if we have significant gaps going into an examination?

We tell you exactly what they are, how serious they are, and what to do about them — in priority order. A gap is not a reason to avoid an assessment. It is precisely the reason to conduct one before your examiner does. Organizations that discover gaps through Thorium's assessment have time to remediate. Organizations that discover them during an examination do not.

Do you provide documentation we can hand directly to an examiner?

Yes. Every compliance engagement includes an examiner-ready evidence package — organized, labeled, and formatted for direct submission to NCUA examiners, OCR auditors, QSAs, or other regulatory reviewers. We write our reports with the examiner's perspective in mind because we understand what they are looking for and how they evaluate what they receive.

Risk Assessments

Is a risk assessment required by our regulatory framework?

Almost certainly yes. A documented risk assessment is one of the most universally required elements across every major framework Thorium works with. FFIEC requires it. HIPAA explicitly mandates an ongoing risk analysis under the Security Rule. CMMC requires it. NIST SP 800-53 requires it. The most common single finding in NCUA and HIPAA examinations is an inadequate or missing risk assessment. If you are subject to any of these frameworks and have not conducted a formal risk assessment recently, this should be your first engagement.

How is Thorium's risk assessment different from completing a questionnaire or using an online tool?

Online tools and questionnaire-based assessments produce a score based on your self-reported answers. Thorium's risk assessments are substantive evaluations — we gather evidence, examine your actual control environment, develop threat scenarios specific to your industry and organizational profile, and produce findings grounded in what we observed rather than what you reported. The difference matters when an examiner asks follow-up questions about your methodology.

How often should a risk assessment be conducted?

Annual risk assessments are the baseline expectation under FFIEC, HIPAA, and CMMC. Beyond the annual cycle, a risk assessment should be triggered by significant changes to your environment — major system deployments, cloud migrations, acquisitions, significant staffing changes in IT or leadership, or following a security incident. Risk is not static and neither should your assessment of it be.

Is a risk assessment required by our regulatory framework?

How is Thorium's risk assessment different from completing a questionnaire?

How often should a risk assessment be conducted?

Cloud Security Configuration Audits

We use a managed IT provider who handles our cloud environment. Do we still need this assessment?

Yes — and this is one of the most important questions we get asked. Managed IT providers are responsible for keeping your systems running. Security configuration is a separate discipline, and many managed providers apply default or vendor-recommended settings rather than security-hardened configurations. The responsibility for the security of your cloud environment ultimately rests with your organization, not your provider. An independent assessment of your cloud configuration is the only way to know whether it meets security best practices — regardless of who manages it day-to-day.

What access do you need to conduct a cloud security audit?

We work from read-only access — we do not require administrative credentials or the ability to make changes to your environment. For Azure and Microsoft 365, a Global Reader role assignment is typically sufficient. For AWS, a read-only IAM policy covers most assessment requirements. We document all access requirements during scoping and work with your IT team to establish the appropriate permissions before the engagement begins.

Do you assess cloud environments used by our vendors or third parties?

Our standard cloud security configuration audit covers your organization's own cloud tenants and subscriptions. Third-party and vendor cloud environments require separate authorization and scoping. If third-party cloud access is a concern — for example, vendors who have access to your Microsoft 365 tenant or AWS environment — we can include that in scope with appropriate agreements in place.

We use a managed IT provider who handles our cloud environment.

Managed IT providers are responsible for keeping your systems running. Security configuration is a separate discipline, and many managed providers apply default or vendor-recommended settings rather than security-hardened configurations. The responsibility for the security of your cloud environment ultimately rests with your organization, not your provider. An independent assessment of your cloud configuration is the only way to know whether it meets security best practices — regardless of who manages it day-to-day.

What access do you need to conduct a cloud security audit?

Do you assess cloud environments used by our vendors or third parties?

We use a managed IT provider. Do we still need this assessment?

What access do you need to conduct a cloud security audit?

Do you assess cloud environments used by our vendors or third parties?