• Minemeld Edl
• Minemeld Edl Inputs
• Minemeld Edl Input Tools

MineMeld is an open source threat feed management system that gathers IP addresses, URLs, and domains which pose a significant network security threat. The threat feed sources can either be free, subscription-based or proprietary. MineMeld re-scans the feeds at regular time intervals and continuously aggregates and updates the set of all threat indicators to be consumed by fierwalls, IDS/IPS, or any other security device.

Input: Threat Feeds. OSINT. Commercial. Organization (CERT, ISAC). Autofocus Processors. IPv4/IPv6 aggregator. URL aggregator. Domain aggregator Outputs. JSON. STIX/TAXII. External Dynamic List (EDL). Elastic Logstash End Point Enforcers Network Enforcers FW, IPS SIEM. Export const txt = 'nn Use the Palo Alto Networks MineMeld integration to manage your MineMeld miners from within Demisto. All commands require then n super adminn n role.nnn Use Casesnnn n Add or remove indicators from a miner.n n n Fetch miners, IP addresses, files, domains, and URLs.n n n Get a list of all your miners.n nnn n NOTEn nnnn n Navigate ton.

MetaFlows now includes MineMeld public threat feeds to augment our existing intelligence sources. The public threat feeds amount to about 200,000 additional indicators updated every few hours. Users also have the ability to add site-specific (either subscription-based or private) MimeMeld sources.

IPv4 and URL/Domain indicators are treated differently.

IPv4 feeds

The default MineMeld IPv4 feeds processed by MetaFlows are below:

MineMeld IPv4 addresses are compiled in a set of IDS/IPS rules designed to alert or block communications to blacklisted addresses. MetaFlows uses a proprietary technique to quickly look through this huge list of addresses (140,000+) and therefore does not require specialized hardware for hi-speed networks.

The MineMeld IPv4 feeds are in the mmreputation.rules configuration file that can be accessed through the existing IDS rule management UI. The feeds are not activated by default but users can activate them in IDS or IPS mode with just a few clicks. If enabled, these rules can be very useful to detect and/or prevent communication to questionable hosts on the Internet.

All the IP addresses are reduced to approximately 40 separate signatures. Each signature corresponds to a specific feed source (for example blocklist_de) or intersections of sources where the IPv4 address is present in more than one source (for example blocklist_de_alienvault.reputation). This decomposition provides additional operational awareness that can be used to prioritize which set of IPs to alert on or block. Enabling or blocking individual signatures therefore affects a dynamically changing set of potentially thousands of IPs updated every few hours that map to a single threat feed or the intersection of multiple threat feeds.

Users also have the option of adding site-specify MineMeld IPv4 feeds to enable additional commercial MineMeld subscriptions independently purchased or other proprietary feeds.

Entering the URL as shown above, will automatically add the custom MineMeld reputation feed into the customer’s configuration and the local rule corresponding to the feed can then be managed as the other public MineMeld feeds.

URL and Domain FeedS

The MineMeld domain and URL feeds processed by MetaFlows are below:

These feeds are used to detect when:

• A user issues an HTTP request to a URL or domain deemed to be malicious or
• A user receives an email containing a malicious URL or link to a malicious domain whether or not the user clicks on the links.

When either of these two conditions occur, a high priority even is generated that can be used to block those specific communications.

There is also a an additional option to enable real time email notification. When bad emails are detected, users also get a warning email instructing them to discard the email.

MineMeld support will automatically be added next time your system self updates or if the sensor software is restarted.

MineMeld is an “extensible Threat Intelligence processing framework and the ‘multi-tool’ of threat indicator feeds. Based on an extremely flexible engine, MineMeld can be used to collect, aggregate and filter indicators from a variety of sources and make them available for consumption to peers or to the Palo Alto Networks security platforms.”

It was recently open-sourced by Palo Alto and can be found on Github.

Essentially it can be used to grab IP/URL/Domain feeds from anywhere on the internet (a miner), aggregate and process the feed or feeds using regex if necessary (a processor) and output them in a format suitable to use in an External Dynamic List object on a Palo Alto firewall.

Technically the outputs can be used for anything you want but they work with dynamic lists on the Palo Alto’s out of the box.

I’ve only used MineMeld for a few weeks but I have a few feeds configured – I’ll go through the configuration of one of them now. It’s pretty straight forward but hopefully it’ll come in handy.

Blocking Tor Exit Nodes

In this example we’ll do the following:

• Configure the tor exit node (miner)
• Configure an aggregator (processor)
• Configure the output in a format suitable for your PAN firewall (output)
• Configure a new External Dynamic List (EDL) object on your Palo to look for the output you created in MineMeld
• Create a new security policy on the firewall to block outbound access to the Tor exit nodes.
• Confirm the EDL object on the firewall is being populated
• Confirm that traffic to Tor exit addresses are indeed being blocked

Minemeld Edl

Let’s get started… if you don’t have MineMeld set-up already then you should probably do that first before continuing! You can download the .ova so you can use it in VMware (I have it set up on VMware workstation at the moment) or install it manually on Ubuntu (installing it manually is probably best for a production environment)

Minemeld Edl Inputs

1. First let’s configure the Tor miner. This essentially sets up a process in MineMeld to go and grab the list of Tor exit nodes. Tor makes this information available publicly. As you can see, the format is not suitable for import just yet.Click Config in MineMeld. You’ll see a bunch of default miners, processors and outputs. I deleted all of them as they weren’t useful for me.
   Click the Add button and give the miner a useful name. From the prototype dropdown select tor.exit_addresses. There are no inputs. Click Save.
2. Now we want to set up a feed aggregator/processor. Click the Add button again and this time choose the processor stdlib.aggregatorIPv4Generic
3. Lastly we want to create an output. This is essentially a clean, formatted version of the raw IP addresses we saw in step 1. Click the Add button and give the output an appropriate name and select stdlib.feedHCGreen from the dropdown. Make sure you select the processor/aggregator as the input.
4. Commit the changes by clicking on the Commit button on the top left of the Config screen. Within a few minutes your Nodes page should look like the below. Don’t forget that if you are blocking the app-id tor on your Palo, MineMeld won’t be able to get the IP address list from the tor web server!If you click the tor-exit-nodes-output, you’ll see a feed base url field with a direct link to the feed which is now hosted on your MineMeld server. This is what we’ll use in the Palo next.
5. Now let’s create an External Dynamic List object on the firewall. Click Objects then External Dynamic List. Click Add and fill in the details – the most important is the feed url which is the one we looked at just above. Click Test Source URL which should report back a success message. If it doesn’t then ensure your Palo can access your MineMeld server).
6. Now we’ll create a security policy that will block all outbound access to this dynamic list; aka Tor exit node IP’s. Create a security policy as you normally would but this time put the new external dynamic list as a destination address. For example:
7. Now we want to make sure the EDL is being populated correctly on the firewall. Log-in to the CLI and run the following command:

   request system external-list show type ip name minemeld-tor-exit-nodes

   You should see something like this if the firewall is successfully pulling the information down from your MineMeld server.

8. Finally… time to test the block list to make sure we’re actually blocking requests to the Tor exit nodes. I attempted to initiate a few requests to a Tor exit node via http, https and ssh. As expected, they were all blocked by the firewall:

Minemeld Edl Input Tools

Related posts: