Repository navigation
[New Integration] HaveIBeenPwned #17352
Description
HaveIBeenPwned (HIBP) is a service that allows users and organization admins to search for usernames and email addresses included in data breaches. Orgs can also use the HIBP API to retrieve breach information for all email addresses using the org's domains.
Useful API endpoints to retrieve
- breacheddomain : A list of email aliases on a domain. Each email alias has a list of breaches it was in
- breaches: This collects a list of all breaches and details of the breach.
- subscribeddomains: All domains the API key is subscribed to. This includes aggregated information such as total accounts and total accounts minus spamlists. It also includes information on the subscription itself, such as renewal date.
Example of breaches:
[
{
"Name": "Adobe",
"Title": "Adobe",
"Domain": "adobe.com",
"BreachDate": "2013-10-04",
"AddedDate": "2013-12-04T00:00:00Z",
"ModifiedDate": "2022-05-15T23:52:49Z",
"PwnCount": 152445165,
"Description": "In October 2013, 153 million Adobe accounts were breached with each containing an internal ID, username, email, <em>encrypted</em> password and a password hint in plain text. The password cryptography was poorly done and many were quickly resolved back to plain text. The unencrypted hints also <a href="http://www.troyhunt.com/2013/11/adobe-credentials-and-serious.html" target="_blank" rel="noopener">disclosed much about the passwords</a> adding further to the risk that hundreds of millions of Adobe customers already faced.",
"LogoPath": "Adobe.png",
"DataClasses": [
"Email addresses",
"Password hints",
"Passwords",
"Usernames"
],
"IsVerified": true,
"IsFabricated": false,
"IsSensitive": false,
"IsRetired": false,
"IsSpamList": false,
"IsMalware": false,
"IsStealerLog": false,
"IsSubscriptionFree": false
},Example of breached domain endpoint:
{
"alias1": [
"Adobe"
],
"alias2": [
"Adobe",
"Gawker",
"Stratfor"
],
"alias3": [
"AshleyMadison"
]
}Uses cases would be enrichment of alerts or events based on users included in breaches, threat intel enrichment, and entity-risk measurement.
I do have a custom integration built with elastic-package for for this. I'm willing to add that in a PR, though I'm not really sure how to go about that with a new integration.
Example dashboard (most panels need the custom enrichment)
Activity
elasticmachine commentedon Feb 26, 2026
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)
P1llus commentedon Mar 6, 2026
Hello @jvalente-salemstate. Thanks for sending in the request! Just taking another look at this and I was curious what a user would expect to be able to configure for those endpoints? For example breacheddomain would need the domain name in question,
I was browsing their API docs a bit, looking what they have available, also saw a notice on the
breacheddomaindocs that:For the
breachesI assume an optional "domains" filter would be useful, as the API exposes it, but its not a required parameter, similarly withincludeUnverified=trueAnother issue is that we usually don't include enrich policies directly in the integration, though we do offer ESQL LOOKUP JOIN now, which can do the same on query time, so you don't have to manage enrich policies.
There is a few small issues with the API that makes it a bit more complex, so there might be a better way around it, it does not offer any date filtering, which means we are collecting all the data each time we run it, and either we duplicate the data on storage, drop duplicates that has not changed, or use a transform bundled in the integration (similar to our other threat intel integrations)
It might be good to see if we can understand what the prefered data schema could look like in the end, for example we do have the possibility to call one API, then based on the response it returns we can call another API for each of the items in a response to gather all the relevant information before we send the data to elastic. If we consider that, maybe there is a combination of multiple API endpoints we could use to collect the most useful parts? We could still keep some of these as useful metadata and for visualization purposes.
Just wanted to get your 2 cents on the topic as a consumer of the product. On the question about contributing integrations we do always provide the possibility to open PR's against our repo but we cannot always promise that it gets merged, but not without reason.
If you do end up going that route just make sure that any data added to it is sanitized or anonymized in some way.
If you want to continue down the issue route instead maybe we can start with the above questions to see how we could provide the best user experience :)
jvalente-salemstate commentedon Mar 10, 2026
Currently, I have it set to pull all of the datasets just once per day, largely because the enrich policy and being able to just set the filter on the dashboard to the last 24 hours. A call could be made to
GET https://haveibeenpwned.com/api/v3/latestbreach, withAddedDateas a cursor to determine if a second query is needed.For that filter, the domain of the breach would need to be known beforehand since it's pulling the breached's site's domain rather than user (e.g. adobe.com instead of elastic.co ) but the breachedomain endpoint would likely benefit from multiple domains here. I've only set it up to work with one, but a loop through a list of domains (or multiple policies) would work. A true/false setting in the policy for
includeVerifiedcould be done though.I'll need to revisit this part. I initially tried using a lookup index, but
elastic-packagewould fail when building/validating. That would also be my preferred method of handling it since it would allow enrichment in far more cases and avoids having to set up a watcher to execute the policy.Regarding duplication. Pulling the full
breachesdataset yields 960 documents per day. This would increase as more breaches are added, but it was a small enough number I didn't put any time into handling duplication. Thebreacheddomainsdoes have a bit more duplication. We're seeing ~47k documents generated each run because a doc is created for each user+breach combo, for about 16k unique email addresses. This would vary much more depending on org size and how many breaches a given user is in.Going with a lookup join would likely help with that since splitting may not be necessary.
If I do submit a PR, should I break it into smaller commits or submit it as-is? At least in terms of ease of review.