数据分析：情报威胁数据分析(1) 郝伟 2021/05/23 [TOC]

1. 1 数据简介简介

从马维士处获得的数据 watch_lab_vul_20190814_20210114.iso 大小约为8G。本文的目标是进行初步的分析，以理解数据的主要结构。此数据来自于安全研究团队：守望者实验室。

2. 2 数据初步分析

根据从马维士处获得的数据 watch_lab_vul_20190814_20210114.iso 大小约为8G，初步分析。 加载后发现其数据主要包括以下：

2021/01/13  10:58        40,593,629 all-15day-actively-20191015.zip
2021/01/13  10:49        17,466,698 all-15day-actively-20191115.zip
2021/01/13  10:41        43,351,510 all-15day-actively-20191215.zip
2021/01/13  10:30        31,994,299 all-15day-actively-20200715.zip
2021/01/13  10:22        28,762,592 all-15day-actively-20200815.zip
...(此处省略若干行)...
2021/01/13  09:33         1,385,256 all-1day-actively-20210107.zip
2021/01/13  09:33           711,436 all-1day-actively-20210108.zip
2021/01/13  09:32           722,372 all-1day-actively-20210109.zip
2021/01/13  09:32         2,359,520 all-1day-actively-20210110.zip
2021/01/13  09:32           629,489 all-1day-actively-20210111.zip
2021/01/13  09:30         4,497,152 all-1day-actively-20210112.zip
2021/01/13  10:50           366,273 all-1day-newly-20190814.zip
2021/01/13  11:02           365,771 all-1day-newly-20191001.zip
2021/01/13  11:02           792,608 all-1day-newly-20191002.zip
...(此处省略若干行)...
2021/01/13  09:32           353,926 all-1day-newly-20210110.zip
2021/01/13  09:32           284,088 all-1day-newly-20210111.zip
2021/01/13  09:31           299,365 all-1day-newly-20210112.zip
2021/01/13  10:59        21,129,746 all-20190304.zip
2021/01/13  10:51        21,874,317 all-20190814.zip
2021/01/13  11:02        23,333,086 all-20191001.zip
...(此处省略若干行)...
2021/01/13  12:18        31,340,313 all-20210109.zip
2021/01/13  15:09        33,018,174 all-20210110.zip
2021/01/13  15:09        28,811,251 all-20210111.zip
2021/01/13  12:17        32,702,914 all-20210112.zip
2021/01/13  10:54       111,576,741 all-30day-actively-20191031.zip
2021/01/13  10:45        61,709,015 all-30day-actively-20191130.zip
2021/01/13  10:34        99,947,666 all-30day-actively-20191231.zip
2021/01/13  10:26        50,621,121 all-30day-actively-20200731.zip
2021/01/13  10:17        50,875,585 all-30day-actively-20200831.zip
2021/01/13  10:08        40,612,473 all-30day-actively-20201031.zip
2021/01/13  09:53        31,977,140 all-30day-actively-20201130.zip
2021/01/13  09:37        57,666,666 all-30day-actively-20201231.zip
2021/01/13  10:54        47,452,025 all-30day-newly-20191031.zip
2021/01/13  10:45        20,437,381 all-30day-newly-20191130.zip
2021/01/13  10:26        12,006,766 all-30day-newly-20200731.zip
2021/01/13  10:17         6,996,820 all-30day-newly-20200831.zip
2021/01/13  10:08        11,406,961 all-30day-newly-20201031.zip
2021/01/13  09:53        10,802,681 all-30day-newly-20201130.zip
2021/01/13  09:37        10,621,794 all-30day-newly-20201231.zip
             793 个文件  8,914,168,200 字节
               0 个目录              0 可用字节

结构随机查看，发现每个压缩包的内容基本相同，于是就选择 S:\Ai.KG\data\all-1day-actively-20190814.zip 的内容进行分析。

解压后，得到以下文件列表：

2019/11/16  23:26            16,991 domain_c2-20190814day-actively.json
2019/11/16  23:26            99,529 domain_reputation-20190814day-actively.json
2019/11/16  23:26           265,186 email_reputation-20190814day-actively.json
2019/11/16  23:26           195,336 email_spamming-20190814day-actively.json
2019/11/16  23:26         2,085,552 hash_reputation-20190814day-actively.json
2019/11/16  23:26            28,015 ip_c2-20190814day-actively.json
2021/05/19  10:41         1,664,846 ip_proxy-20190814day-actively.json
2021/05/19  10:41         8,252,182 ip_reputation-20190814day-actively.json
2021/05/19  10:41           566,742 ip_spamming-20190814day-actively.json
2021/05/19  10:41           293,689 ip_tor-20190814day-actively.json
2021/05/19  10:41               719 url_c2-20190814day-actively.json
2021/05/19  10:41           657,852 url_phishing-20190814day-actively.json
2021/05/19  10:41         2,585,928 url_reputation-20190814day-actively.json

打开每个文件查看，发现每个文件中并不是一个独立的Json数据，而是每行是一个json数据，即有多少行就有多少个json对象，如打开 domain_c2-20190814day-actively 文件，发现以下内容：

{"attackerInfo":[],"whois":[],"basicInfo":{"lastTime":"2019-10-22 17:12:00","firstTime":"2018-08-01 00:49:13","total":106,"data":"applegarden.net","attackAction":["c2"],"dataType":"domain","attackInProtocol":["dns"],"malwareClass":["pizd"],"tags":"="},"threatIntelligence":[{"level":75,"activeTime":"2019-10-22 17:12:00","channel":"domain_reputation"},{"level":75,"activeTime":"2019-10-22 17:12:00","channel":"domain_c2"}],"linkedAnalysis":[]}
{"attackerInfo":[],"whois":[],"basicInfo":{"lastTime":"2019-10-22 17:12:00","firstTime":"2018-08-01 00:49:13","total":148,"data":"beautyforward.net","attackAction":["c2"],"dataType":"domain","attackInProtocol":["dns"],"malwareClass":["pizd"],"tags":"="},"threatIntelligence":[{"level":75,"activeTime":"2019-10-22 17:12:00","channel":"domain_reputation"},{"level":75,"activeTime":"2019-10-22 17:12:00","channel":"domain_c2"}],"linkedAnalysis":[]}
...(以下省略若干行)...

3. 3 两个问题

为了导入数据，必需对数据格式进行全面分析，所以现在有这样的2个问题：

3.1. 3.1 问题1：一个文件中的数据格式是否全部一样？

为了回答这个问题，首先，选择一个较小的文件进行分析，如 email_apt-20210112.json。

首先，部分代码参见 对json文件进行动态遍历以生成所有键的完整路径，定义以下两个函数：

import json

def analyze(json_data, paths, cur_path):
    if isinstance(json_data, dict):
        for key in json_data:
            new_path='/'.join(cur_path) + '/' + key
            if paths.count(new_path) == 0:
                paths.append(new_path)
            cur_path.append(key)
            analyze(json_data[key], paths, cur_path)
            del cur_path[-1]
    elif isinstance(json_data, list):
          cur_path.append('[*]')
          for i in range(len(json_data)):
              analyze(json_data[i], paths, cur_path)
          del cur_path[-1]
    return (paths, cur_path)

def analyze_json_path(json_data):
    paths=[]
    cur_path=['.']
    paths, cur_path = analyze(json_data, paths, cur_path)
    return paths

def get_json_path(jsonfile):
    jdata = json.loads(jsonfile)
    paths = []
    for path in analyze_json_path(jdata):
        paths.append(path)
    return paths 

filepath = r'S:\Ai.KG\data\all-1day-actively-20190814\domain_c2-20190814day-actively.json';
allpaths={}
try: 
    fp = open(filepath, 'r', encoding='utf-8')
    for line in fp.readlines():
        json_data = json.loads(line)
        paths = analyze_json_path(json_data)
        for path in paths:
            if path not in allpaths:
                allpaths[path] = 0
            allpaths[path] = allpaths[path] + 1
except Exception as e:
    print("error:", filepath)
    print(e) 
for path in allpaths:
    print(allpaths[path], path)

运行结果如下：

38 ./attackerInfo
38 ./whois
38 ./basicInfo
38 ./basicInfo/lastTime
38 ./basicInfo/firstTime
38 ./basicInfo/total
38 ./basicInfo/data
38 ./basicInfo/attackAction
38 ./basicInfo/dataType
38 ./basicInfo/attackInProtocol
38 ./basicInfo/malwareClass
38 ./basicInfo/tags
38 ./threatIntelligence
38 ./threatIntelligence/[*]/level
38 ./threatIntelligence/[*]/activeTime
38 ./threatIntelligence/[*]/channel
38 ./linkedAnalysis

根据输出结果，可见每一行的内容都相同。

3.2. 3.2 问题2：所有的文件格式是否都是一样的？

在回答了问题1，现在再分析其他文件以回答问题2。在代码中添加文件夹遍历函数，得到以下内容：

dirname=R'S:\Ai.KG\data\all-1day-actively-20190814'
for filename in os.listdir(dirname):
    print(filename.center(70, '*'))
    filepath = os.path.join(dirname, filename)
    allpaths={}
    try: 
        fp = open(filepath, 'r', encoding='utf-8')
        for line in fp.readlines():
            json_data = json.loads(line)
            paths = analyze_json_path(json_data)
            for path in paths:
                if path not in allpaths:
                    allpaths[path] = 0
                allpaths[path] = allpaths[path] + 1
    except Exception as e:
        print("error:", filepath)
        print(e) 
    for path in allpaths:
        print(allpaths[path], path)

代码行后的结果

*****************domain_c2-20190814day-actively.json******************
38 ./attackerInfo
38 ./whois
38 ./basicInfo
38 ./basicInfo/lastTime
38 ./basicInfo/firstTime
38 ./basicInfo/total
38 ./basicInfo/data
38 ./basicInfo/attackAction
38 ./basicInfo/dataType
38 ./basicInfo/attackInProtocol
38 ./basicInfo/malwareClass
38 ./basicInfo/tags
38 ./threatIntelligence
38 ./threatIntelligence/[*]/level
38 ./threatIntelligence/[*]/activeTime
38 ./threatIntelligence/[*]/channel
38 ./linkedAnalysis
*************domain_reputation-20190814day-actively.json**************
265 ./attackerInfo
265 ./whois
265 ./basicInfo
265 ./basicInfo/lastTime
265 ./basicInfo/firstTime
265 ./basicInfo/total
265 ./basicInfo/data
265 ./basicInfo/attackAction
265 ./basicInfo/dataType
265 ./basicInfo/attackInProtocol
265 ./basicInfo/malwareClass
265 ./basicInfo/tags
265 ./threatIntelligence
265 ./threatIntelligence/[*]/level
265 ./threatIntelligence/[*]/activeTime
265 ./threatIntelligence/[*]/channel
265 ./linkedAnalysis
1 ./threatIntelligence/[*]/domain
1 ./threatIntelligence/[*]/source
1 ./threatIntelligence/[*]/saveTime
1 ./threatIntelligence/[*]/class
1 ./threatIntelligence/[*]/tags
1 ./threatIntelligence/[*]/ip
**************email_reputation-20190814day-actively.json**************
618 ./basicInfo
618 ./basicInfo/lastTime
618 ./basicInfo/firstTime
618 ./basicInfo/total
618 ./basicInfo/data
618 ./basicInfo/attackAction
618 ./basicInfo/dataType
618 ./basicInfo/attackInProtocol
618 ./basicInfo/malwareClass
618 ./basicInfo/tags
618 ./threatIntelligence
618 ./threatIntelligence/[*]/level
618 ./threatIntelligence/[*]/activeTime
618 ./threatIntelligence/[*]/channel
618 ./linkedAnalysis
124 ./threatIntelligence/[*]/ip
1 ./threatIntelligence/[*]/source
1 ./threatIntelligence/[*]/saveTime
1 ./threatIntelligence/[*]/email
***************email_spamming-20190814day-actively.json***************
452 ./basicInfo
452 ./basicInfo/lastTime
452 ./basicInfo/firstTime
452 ./basicInfo/total
452 ./basicInfo/data
452 ./basicInfo/attackAction
452 ./basicInfo/dataType
452 ./basicInfo/attackInProtocol
452 ./basicInfo/malwareClass
452 ./basicInfo/tags
452 ./threatIntelligence
452 ./threatIntelligence/[*]/level
452 ./threatIntelligence/[*]/activeTime
452 ./threatIntelligence/[*]/channel
452 ./linkedAnalysis
23 ./threatIntelligence/[*]/ip
1 ./threatIntelligence/[*]/source
1 ./threatIntelligence/[*]/saveTime
1 ./threatIntelligence/[*]/email
**************hash_reputation-20190814day-actively.json***************
4616 ./basicInfo
4616 ./basicInfo/firstTime
4616 ./basicInfo/lastTime
4616 ./basicInfo/total
4616 ./basicInfo/data
4616 ./basicInfo/attackAction
4616 ./basicInfo/dataType
4616 ./basicInfo/origin
4616 ./basicInfo/origin/sha1
4616 ./basicInfo/origin/sha256
4616 ./basicInfo/origin/md5
4616 ./basicInfo/malwareClass
4616 ./basicInfo/tags
4616 ./threatIntelligence
4616 ./threatIntelligence/[*]/level
4616 ./threatIntelligence/[*]/activeTime
4616 ./threatIntelligence/[*]/channel
4616 ./linkedAnalysis
*******************ip_c2-20190814day-actively.json********************
45 ./attackerInfo
45 ./whois
45 ./basicInfo
45 ./basicInfo/lastTime
45 ./basicInfo/firstTime
45 ./basicInfo/total
45 ./basicInfo/data
45 ./basicInfo/attackAction
45 ./basicInfo/dataType
45 ./basicInfo/attackInProtocol
45 ./basicInfo/malwareClass
45 ./basicInfo/location
45 ./basicInfo/location/cityName
45 ./basicInfo/location/countryCode
45 ./basicInfo/location/latitude
45 ./basicInfo/location/countryName
45 ./basicInfo/location/provinceName
45 ./basicInfo/location/longitude
45 ./basicInfo/tags
45 ./threatIntelligence
45 ./threatIntelligence/[*]/level
45 ./threatIntelligence/[*]/activeTime
45 ./threatIntelligence/[*]/channel
45 ./linkedAnalysis
6 ./threatIntelligence/[*]/domain
2 ./threatIntelligence/[*]/ip
2 ./threatIntelligence/[*]/source
2 ./threatIntelligence/[*]/saveTime
2 ./threatIntelligence/[*]/class
******************ip_proxy-20190814day-actively.json******************
error: S:\Ai.KG\data\all-1day-actively-20190814\ip_proxy-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
***************ip_reputation-20190814day-actively.json****************
error: S:\Ai.KG\data\all-1day-actively-20190814\ip_reputation-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
****************ip_spamming-20190814day-actively.json*****************
error: S:\Ai.KG\data\all-1day-actively-20190814\ip_spamming-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
*******************ip_tor-20190814day-actively.json*******************
error: S:\Ai.KG\data\all-1day-actively-20190814\ip_tor-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
*******************url_c2-20190814day-actively.json*******************
error: S:\Ai.KG\data\all-1day-actively-20190814\url_c2-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
****************url_phishing-20190814day-actively.json****************
error: S:\Ai.KG\data\all-1day-actively-20190814\url_phishing-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
***************url_reputation-20190814day-actively.json***************
error: S:\Ai.KG\data\all-1day-actively-20190814\url_reputation-20190814day-actively.json
Expecting property name enclosed in double quotes: line 2 column 1 (char 2)

由此可见，部分文件中有不内的内容，同时还有几个文件格式不一样。 经分析发现，不能转换的文件是因为两种情况：

  • 内容本身就是一个完整的json文件； 处理方法：按单独的json文件处理。
  • 内容是由多个文件组成，但是一段json已经格式化好了，分成多个行。 根据 \n}\n{\n 替换为 \n},\n{\n 同时加上开头和结尾使其成为一个独立的文件。
import json, os


def load_json(json_filepath):
    data = []
    with open(json_filepath, 'r', encoding='utf-8') as load_f: 
         data =  json.load(load_f)
    return data

def save_file(content, filename):
    fp = open(filename, 'w+', encoding='utf-8')
    fp.write(content)
    fp.close()

def analyze(json_data, paths, cur_path):
    if isinstance(json_data, dict):
        for key in json_data:
            new_path='/'.join(cur_path) + '/' + key
            if paths.count(new_path) == 0:
                paths.append(new_path)
            cur_path.append(key)
            analyze(json_data[key], paths, cur_path)
            del cur_path[-1]
    elif isinstance(json_data, list):
          cur_path.append('[*]')
          for i in range(len(json_data)):
              analyze(json_data[i], paths, cur_path)
          del cur_path[-1]
    return (paths, cur_path)

def analyze_json_path(json_data):
    paths=[]
    cur_path=['.']
    paths, cur_path = analyze(json_data, paths, cur_path)
    return paths

def get_json_path(jsonfile):
    jdata = load_json(jsonfile)
    paths = []
    for path in analyze_json_path(jdata):
        paths.append(path)
    return paths


dirname=R'S:\Ai.KG\data\all-1day-actively-20190814'

jsonfilelist=[]
for filename in os.listdir(dirname):
    filepath = os.path.join(dirname, filename)
    allpaths={}
    try: 
        fp = open(filepath, 'r', encoding='utf-8')
        for line in fp.readlines():
            json_data = json.loads(line)
            paths = analyze_json_path(json_data)
            for path in paths:
                if path not in allpaths:
                    allpaths[path] = 0
                allpaths[path] = allpaths[path] + 1
        fp.close()
        print(filename.center(70, '*'))
    except Exception as e:
        jsonfilelist.append(filepath)
        try:
            paths = get_json_path(filepath)
            for path in paths:
                if path not in allpaths:
                    allpaths[path] = 0
                allpaths[path] = allpaths[path] + 1
        except Exception as e1:
            pass
    for path in allpaths:
        print(allpaths[path], path)



for filepath in jsonfilelist:
    content = ''
    try:
        with open(filepath, 'r', encoding='utf-8') as fp: 
            content = ''.join(fp.readlines())
        content = '{ "items": [' + content.replace('\n}\n{\n', '\n},\n{\n') + ']}'
        #save_file(content, filepath + '.txt')
        items = json.loads(content)
        allpaths = {}
        for item in items['items']:
            paths = analyze_json_path(item)
            for path in paths:
                if path not in allpaths:
                    allpaths[path] = 0
                allpaths[path] = allpaths[path] + 1

        paths = analyze_json_path(json.loads(content))
        print('*' * 80)
        print(filepath)
#        for path in paths:
#            print(path.replace('./items/[*]', '.'))
        for path in allpaths:
            print(allpaths[path], path)
    except:
        print('** error **', filepath)

(base) C:\Gitee.com\MyNotes> cmd /C "C:\ProgramData\Anaconda3\python.exe c:\Users\hwaus\.vscode\extensions\ms-python.python-2021.4.765268190\pythonFiles\lib\python\debugpy\launcher 49813 -- c:\Gitee.com\MyNotes\zTemSourceCodes\20210524_thread_info_path_analysis.py "
*****************domain_c2-20190814day-actively.json******************
38 ./attackerInfo
38 ./whois
38 ./basicInfo
38 ./basicInfo/lastTime
38 ./basicInfo/firstTime
38 ./basicInfo/total
38 ./basicInfo/data
38 ./basicInfo/attackAction
38 ./basicInfo/dataType
38 ./basicInfo/attackInProtocol
38 ./basicInfo/malwareClass
38 ./basicInfo/tags
38 ./threatIntelligence
38 ./threatIntelligence/[*]/level
38 ./threatIntelligence/[*]/activeTime
38 ./threatIntelligence/[*]/channel
38 ./linkedAnalysis
*************domain_reputation-20190814day-actively.json**************
265 ./attackerInfo
265 ./whois
265 ./basicInfo
265 ./basicInfo/lastTime
265 ./basicInfo/firstTime
265 ./basicInfo/total
265 ./basicInfo/data
265 ./basicInfo/attackAction
265 ./basicInfo/dataType
265 ./basicInfo/attackInProtocol
265 ./basicInfo/malwareClass
265 ./basicInfo/tags
265 ./threatIntelligence
265 ./threatIntelligence/[*]/level
265 ./threatIntelligence/[*]/activeTime
265 ./threatIntelligence/[*]/channel
265 ./linkedAnalysis
1 ./threatIntelligence/[*]/domain
1 ./threatIntelligence/[*]/source
1 ./threatIntelligence/[*]/saveTime
1 ./threatIntelligence/[*]/class
1 ./threatIntelligence/[*]/tags
1 ./threatIntelligence/[*]/ip
**************email_reputation-20190814day-actively.json**************
618 ./basicInfo
618 ./basicInfo/lastTime
618 ./basicInfo/firstTime
618 ./basicInfo/total
618 ./basicInfo/data
618 ./basicInfo/attackAction
618 ./basicInfo/dataType
618 ./basicInfo/attackInProtocol
618 ./basicInfo/malwareClass
618 ./basicInfo/tags
618 ./threatIntelligence
618 ./threatIntelligence/[*]/level
618 ./threatIntelligence/[*]/activeTime
618 ./threatIntelligence/[*]/channel
618 ./linkedAnalysis
124 ./threatIntelligence/[*]/ip
1 ./threatIntelligence/[*]/source
1 ./threatIntelligence/[*]/saveTime
1 ./threatIntelligence/[*]/email
***************email_spamming-20190814day-actively.json***************
452 ./basicInfo
452 ./basicInfo/lastTime
452 ./basicInfo/firstTime
452 ./basicInfo/total
452 ./basicInfo/data
452 ./basicInfo/attackAction
452 ./basicInfo/dataType
452 ./basicInfo/attackInProtocol
452 ./basicInfo/malwareClass
452 ./basicInfo/tags
452 ./threatIntelligence
452 ./threatIntelligence/[*]/level
452 ./threatIntelligence/[*]/activeTime
452 ./threatIntelligence/[*]/channel
452 ./linkedAnalysis
23 ./threatIntelligence/[*]/ip
1 ./threatIntelligence/[*]/source
1 ./threatIntelligence/[*]/saveTime
1 ./threatIntelligence/[*]/email
**************hash_reputation-20190814day-actively.json***************
4616 ./basicInfo
4616 ./basicInfo/firstTime
4616 ./basicInfo/lastTime
4616 ./basicInfo/total
4616 ./basicInfo/data
4616 ./basicInfo/attackAction
4616 ./basicInfo/dataType
4616 ./basicInfo/origin
4616 ./basicInfo/origin/sha1
4616 ./basicInfo/origin/sha256
4616 ./basicInfo/origin/md5
4616 ./basicInfo/malwareClass
4616 ./basicInfo/tags
4616 ./threatIntelligence
4616 ./threatIntelligence/[*]/level
4616 ./threatIntelligence/[*]/activeTime
4616 ./threatIntelligence/[*]/channel
4616 ./linkedAnalysis
*******************ip_c2-20190814day-actively.json********************
45 ./attackerInfo
45 ./whois
45 ./basicInfo
45 ./basicInfo/lastTime
45 ./basicInfo/firstTime
45 ./basicInfo/total
45 ./basicInfo/data
45 ./basicInfo/attackAction
45 ./basicInfo/dataType
45 ./basicInfo/attackInProtocol
45 ./basicInfo/malwareClass
45 ./basicInfo/location
45 ./basicInfo/location/cityName
45 ./basicInfo/location/countryCode
45 ./basicInfo/location/latitude
45 ./basicInfo/location/countryName
45 ./basicInfo/location/provinceName
45 ./basicInfo/location/longitude
45 ./basicInfo/tags
45 ./threatIntelligence
45 ./threatIntelligence/[*]/level
45 ./threatIntelligence/[*]/activeTime
45 ./threatIntelligence/[*]/channel
45 ./linkedAnalysis
6 ./threatIntelligence/[*]/domain
2 ./threatIntelligence/[*]/ip
2 ./threatIntelligence/[*]/source
2 ./threatIntelligence/[*]/saveTime
2 ./threatIntelligence/[*]/class
1 ./attackerInfo
1 ./whois
1 ./basicInfo
1 ./basicInfo/lastTime
1 ./basicInfo/firstTime
1 ./basicInfo/total
1 ./basicInfo/data
1 ./basicInfo/dataType
1 ./basicInfo/attackAction
1 ./basicInfo/attackInProtocol
1 ./basicInfo/malwareClass
1 ./basicInfo/tags
1 ./threatIntelligence
1 ./threatIntelligence/[*]/level
1 ./threatIntelligence/[*]/activeTime
1 ./threatIntelligence/[*]/channel
1 ./linkedAnalysis
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\ip_proxy-20190814day-actively.json
1464 ./attackerInfo
1464 ./whois
1464 ./basicInfo
1464 ./basicInfo/lastTime
1464 ./basicInfo/firstTime
1464 ./basicInfo/total
1464 ./basicInfo/data
1464 ./basicInfo/attackAction
1464 ./basicInfo/dataType
1464 ./basicInfo/attackInProtocol
1464 ./basicInfo/malwareClass
1464 ./basicInfo/location
1464 ./basicInfo/location/cityName
1464 ./basicInfo/location/countryCode
1464 ./basicInfo/location/latitude
1464 ./basicInfo/location/countryName
1464 ./basicInfo/location/provinceName
1464 ./basicInfo/location/longitude
1464 ./basicInfo/tags
1464 ./threatIntelligence
1464 ./threatIntelligence/[*]/level
1464 ./threatIntelligence/[*]/port
1464 ./threatIntelligence/[*]/activeTime
1464 ./threatIntelligence/[*]/channel
1464 ./threatIntelligence/[*]/anonymity
1464 ./threatIntelligence/[*]/type
1464 ./linkedAnalysis
244 ./threatIntelligence/[*]/email
1 ./threatIntelligence/[*]/exit
1 ./threatIntelligence/[*]/server
1 ./threatIntelligence/[*]/ORPort
1 ./threatIntelligence/[*]/DIRPort
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\ip_proxy-20190814day-actively1.josn
1464 ./attackerInfo
1464 ./whois
1464 ./basicInfo
1464 ./basicInfo/lastTime
1464 ./basicInfo/firstTime
1464 ./basicInfo/total
1464 ./basicInfo/data
1464 ./basicInfo/attackAction
1464 ./basicInfo/dataType
1464 ./basicInfo/attackInProtocol
1464 ./basicInfo/malwareClass
1464 ./basicInfo/location
1464 ./basicInfo/location/cityName
1464 ./basicInfo/location/countryCode
1464 ./basicInfo/location/latitude
1464 ./basicInfo/location/countryName
1464 ./basicInfo/location/provinceName
1464 ./basicInfo/location/longitude
1464 ./basicInfo/tags
1464 ./threatIntelligence
1464 ./threatIntelligence/[*]/level
1464 ./threatIntelligence/[*]/port
1464 ./threatIntelligence/[*]/activeTime
1464 ./threatIntelligence/[*]/channel
1464 ./threatIntelligence/[*]/anonymity
1464 ./threatIntelligence/[*]/type
1464 ./linkedAnalysis
244 ./threatIntelligence/[*]/email
1 ./threatIntelligence/[*]/exit
1 ./threatIntelligence/[*]/server
1 ./threatIntelligence/[*]/ORPort
1 ./threatIntelligence/[*]/DIRPort
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\ip_reputation-20190814day-actively.json
9162 ./attackerInfo
9162 ./whois
9162 ./basicInfo
9162 ./basicInfo/lastTime
9162 ./basicInfo/firstTime
9162 ./basicInfo/total
9162 ./basicInfo/data
9162 ./basicInfo/attackAction
9162 ./basicInfo/dataType
9162 ./basicInfo/attackInProtocol
9162 ./basicInfo/malwareClass
9162 ./basicInfo/location
9162 ./basicInfo/location/cityName
9162 ./basicInfo/location/countryCode
9162 ./basicInfo/location/latitude
9162 ./basicInfo/location/countryName
9162 ./basicInfo/location/provinceName
9162 ./basicInfo/location/longitude
9162 ./basicInfo/tags
9162 ./threatIntelligence
9162 ./threatIntelligence/[*]/level
9162 ./threatIntelligence/[*]/activeTime
9162 ./threatIntelligence/[*]/channel
9162 ./linkedAnalysis
301 ./threatIntelligence/[*]/port
300 ./threatIntelligence/[*]/anonymity
300 ./threatIntelligence/[*]/type
157 ./threatIntelligence/[*]/email
31 ./threatIntelligence/[*]/exit
31 ./threatIntelligence/[*]/server
31 ./threatIntelligence/[*]/ORPort
31 ./threatIntelligence/[*]/DIRPort
10 ./threatIntelligence/[*]/ip
10 ./threatIntelligence/[*]/source
10 ./threatIntelligence/[*]/saveTime
2 ./threatIntelligence/[*]/class
8 ./threatIntelligence/[*]/domain
8 ./threatIntelligence/[*]/tags
1 ./threatIntelligence/[*]/description
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\ip_spamming-20190814day-actively.json
495 ./attackerInfo
495 ./whois
495 ./basicInfo
495 ./basicInfo/lastTime
495 ./basicInfo/firstTime
495 ./basicInfo/total
495 ./basicInfo/data
495 ./basicInfo/attackAction
495 ./basicInfo/dataType
495 ./basicInfo/attackInProtocol
495 ./basicInfo/malwareClass
495 ./basicInfo/location
495 ./basicInfo/location/cityName
495 ./basicInfo/location/countryCode
495 ./basicInfo/location/latitude
495 ./basicInfo/location/countryName
495 ./basicInfo/location/provinceName
495 ./basicInfo/location/longitude
495 ./basicInfo/tags
495 ./threatIntelligence
495 ./threatIntelligence/[*]/level
495 ./threatIntelligence/[*]/activeTime
495 ./threatIntelligence/[*]/channel
495 ./linkedAnalysis
80 ./threatIntelligence/[*]/port
80 ./threatIntelligence/[*]/anonymity
80 ./threatIntelligence/[*]/type
89 ./threatIntelligence/[*]/email
32 ./threatIntelligence/[*]/exit
32 ./threatIntelligence/[*]/server
32 ./threatIntelligence/[*]/ORPort
32 ./threatIntelligence/[*]/DIRPort
7 ./threatIntelligence/[*]/ip
7 ./threatIntelligence/[*]/source
7 ./threatIntelligence/[*]/saveTime
7 ./threatIntelligence/[*]/tags
2 ./threatIntelligence/[*]/reportDesc
2 ./threatIntelligence/[*]/reportName
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\ip_tor-20190814day-actively.json
307 ./attackerInfo
307 ./whois
307 ./basicInfo
307 ./basicInfo/lastTime
307 ./basicInfo/firstTime
307 ./basicInfo/total
307 ./basicInfo/data
307 ./basicInfo/attackAction
307 ./basicInfo/dataType
307 ./basicInfo/attackInProtocol
307 ./basicInfo/malwareClass
307 ./basicInfo/location
307 ./basicInfo/location/cityName
307 ./basicInfo/location/countryCode
307 ./basicInfo/location/latitude
307 ./basicInfo/location/countryName
307 ./basicInfo/location/provinceName
307 ./basicInfo/location/longitude
307 ./basicInfo/tags
307 ./threatIntelligence
307 ./threatIntelligence/[*]/exit
307 ./threatIntelligence/[*]/server
307 ./threatIntelligence/[*]/level
307 ./threatIntelligence/[*]/activeTime
307 ./threatIntelligence/[*]/channel
307 ./threatIntelligence/[*]/ORPort
307 ./threatIntelligence/[*]/DIRPort
307 ./linkedAnalysis
7 ./threatIntelligence/[*]/email
4 ./threatIntelligence/[*]/port
4 ./threatIntelligence/[*]/anonymity
4 ./threatIntelligence/[*]/type
2 ./threatIntelligence/[*]/ip
2 ./threatIntelligence/[*]/source
2 ./threatIntelligence/[*]/saveTime
2 ./threatIntelligence/[*]/tags
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\url_c2-20190814day-actively.json
1 ./attackerInfo
1 ./whois
1 ./basicInfo
1 ./basicInfo/lastTime
1 ./basicInfo/firstTime
1 ./basicInfo/total
1 ./basicInfo/data
1 ./basicInfo/dataType
1 ./basicInfo/attackAction
1 ./basicInfo/attackInProtocol
1 ./basicInfo/malwareClass
1 ./basicInfo/tags
1 ./threatIntelligence
1 ./threatIntelligence/[*]/level
1 ./threatIntelligence/[*]/activeTime
1 ./threatIntelligence/[*]/channel
1 ./linkedAnalysis
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\url_phishing-20190814day-actively.json
795 ./attackerInfo
795 ./whois
795 ./basicInfo
795 ./basicInfo/lastTime
795 ./basicInfo/firstTime
795 ./basicInfo/total
795 ./basicInfo/data
795 ./basicInfo/dataType
795 ./basicInfo/attackAction
795 ./basicInfo/attackInProtocol
795 ./basicInfo/malwareClass
795 ./basicInfo/tags
795 ./threatIntelligence
795 ./threatIntelligence/[*]/level
795 ./threatIntelligence/[*]/activeTime
795 ./threatIntelligence/[*]/channel
792 ./threatIntelligence/[*]/target
795 ./linkedAnalysis
********************************************************************************
S:\Ai.KG\data\all-1day-actively-20190814\url_reputation-20190814day-actively.json
4036 ./attackerInfo
4036 ./whois
4036 ./basicInfo
4036 ./basicInfo/lastTime
4036 ./basicInfo/firstTime
4036 ./basicInfo/total
4036 ./basicInfo/data
4036 ./basicInfo/dataType
4036 ./basicInfo/attackAction
4036 ./basicInfo/attackInProtocol
4036 ./basicInfo/malwareClass
4036 ./basicInfo/tags
4036 ./threatIntelligence
4036 ./threatIntelligence/[*]/level
4036 ./threatIntelligence/[*]/activeTime
4036 ./threatIntelligence/[*]/channel
792 ./threatIntelligence/[*]/target
4036 ./linkedAnalysis

4. 4 结论

数据初步处理已经完成，待进一步分析。

results matching ""

    No results matching ""