社交网络文本特征提取方法研究

社交网络文本特征提取方法研究(论文14000字)
摘要：以社交网络文本为载体的各种应用发展如火如荼，研究文本特征与分类对提取重要信息有极大的价值。本文主要介绍了常用特征选择算法以及特征表示方法，简单介绍了SVM、KNN相关的基本原理、优缺点和分类算法的评价指标。
在互信息特性选择函数方面，具体描述了其面对文本的处理流程、缺点以及优化的改进。针对其在没有很好平衡正负相关特征方面的弱势，引入一种平衡权重属性因子和特征差异因子弥补其不足。
实验阶段主要描述了具体的流程：进行分词处理，去停用字词，使用各种特征选择算法包括优化的互信息，用tf-idf进行加权。在SVM与KNN两种分类算法下，根据评价指标，比较所有特征选择算法的优劣性。实验表明：优化的互信息特征选择具有很好地性能，且在SVM分类算法下比KNN好，此实验证明了其有效性。
关键词：社交网络文本；互信息；正负相关特征；SVM；KNN

Research on text feature extraction in social network
Abstract:The development of various applications based on social network text is in full swing. Studying text features and classifications is of great value to extract important information. This paper mainly introduces the common feature selection algorithms and feature representation methods, and introduces the basic principles, advantages and disadvantages of SVM and KNN, and the evaluation indexes of classification algorithms.
In the aspect of mutual information feature selection function, it describes its processing flow, shortcomings and optimization improvements. In view of its weakness in not balancing the positive and negative correlation characteristics, a balance weight attribute factor and feature difference factor are introduced to make up for its deficiency.
The experimental stage mainly describes the specific process: the word segmentation processing, to disuse words, using various feature selection algorithms, including optimized mutual information, and weighted with TF-IDF. Under the two classification algorithms of SVM and KNN, we compare the merits and demerits of all the feature selection algorithms according to the evaluation index. Experiments show that the optimized mutual information feature selection has good performance and is better than KNN under the SVM classification algorithm. This experiment proves its validity.
Key words：social network text；mutual information；positive and negative correlation characteristics；svm；knn；

目录
1绪论    1
1.1课题研究背景及意义    1
1.2文本特征选择的研究现状    1
1.2.1国外研究现状    2
1.2.2国内研究现状    2
1.3论文的组织结构    3
2社交网络文本特征选择方法综述    3
2.1 社交网络文本特点    3
2.2 社交网络文本常用特征    4
2.3社交网络文本常用特征选择方法    5
2.3.1基尼指数特征选择法    5
2.3.2信息增益特征选择法    6
2.3.3文本证据权特征选择法    6
2.3.4 文档频数特征选择法    7
2.3.5 卡方检验特征选择法    7
2.4 本章总结    8
3基于互信息的文本特征选择算法及其改进    8
3.1传统互信息文本特征选择算法描述    8
3.2互信息特征选择的缺点    9
3.3针对互信息特征选择过程的改进    10
3.3.1 平衡权重属性因子    10
3.3.2 特征差异因子    10
3.4本章总结    11
4常用文本分类算法及评价指标    11
4.1 SVM分类算法    11
4.1.1 SVM算法原理    11
4.1.2 SVN优点与缺点    13
4.2 KNN分类算法    14
4.2.1 KNN算法原理    14
4.2.2 KNN优点与缺点    15
4.3 常用文本分类评价指标    15
5社交网络文本特征选择效果比较及结果分析    16
5.1实验思路与评价指标    16
5.1.1分词处理    16
5.1.2去停用字词    16
5.1.3文本的特征加权表示    17
5.2实验环境    18
5.3实验结果分析    19
5.3.1 中文分词与转码    19
5.3.2特征选择算法对比    20
5.4 本章总结    21
6总结与展望    21
参考文献    22
致谢    24