基于云平台的遥感大数据管理方法(任务书,开题报告,论文13000字)
摘要
目前，遥感数据的获取方式越来越多样化，但主流趋势仍是通过政府或私营企业进行遥感影像数据采集整理，并将数据发布云服务器上，用户可通过浏览器登录相应网址并进行免费或付费下载。而随着互联网技术的深入发展，人们不仅逐渐适应这种服务方式，也逐渐接受这种理念。目前各个卫星都有各自的专门代理平台，但是基于云平台的数据共享方案仍处于发展阶段，彼此间缺乏基于共享的遥感数据，和成熟的专业处理数据共享的云平台
基于这个现状，本文提出一种基于Hadoop集群的遥感数据云端存储并共享的方案，并与之相应地开发了一个小型的系统进行模拟。该平台利用Hadoop分布式文件系统——HDFS的大量性、高效性的特点，可用于存储内存占量巨大的遥感影像，同时利用其高效性特点可实现影像的快速上传和下载。本项目结合MySQL数据库，利用MySQL数据库高效便捷的数据处理功能，实现数据检索的高效性。除此之外，还在数据共享云平台基础上进行延伸，提出了一种多Agent的任务管理功能，以供用户进行任务分解后将不同的难题交予各擅长该领域的人员进行解决，最后整合后完成总体复杂任务。
本系统是基于云服务的理念进行开发的，对于云服务与遥感数据管理进行了有机结合，同时基于问题解决的目的，更延伸了任务管理的机制，对于其他人员的后续研究有明显积极意义。
关键词：Hadoop集群、云平台、数据共享、任务管理
Abstract
At present, the acquisition of remote sensing data is becoming more and more diversified, but the mainstream trend is still to collect and organize remote sensing image data through the government or private enterprises, and release the data to the cloud server. Users can log in the corresponding website through the browser and download the data for free or for payment.With the in-depth development of Internet technology, people not only gradually adapt to this service mode, but also gradually accept this idea. However, there is no effective cloud-based data sharing scheme in the market at present, let alone a platform that can meet the needs of data sharing between users.
Based on this situation, this paper proposes a scheme of remote sensing data cloud storage and sharing based on Hadoop cluster, and accordingly develops a small system for simulation.This platform makes use of the large quantity and high efficiency of Hadoop distributed file system -- HDFS, which can be used to store remote sensing images with a large amount of memory and realize the fast uploading and downloading of images with its high efficiency.Combined with MySQL database, this project utilizes the efficient and convenient data processing function of MySQL database to realize the high efficiency of data retrieval.In addition, it also extends the data sharing cloud platform, and proposes a multi-agent task management function for users to solve different problems after task decomposition to people who are good at this field, and finally complete the overall complex tasks after integration.
This system is developed based on the concept of cloud service, which organically combines cloud service and remote sensing data management. At the same time, based on the purpose of problem solving, it extends the mechanism of task management and has obvious positive significance for the follow-up research of other personnel.
Key Words：Hadoop cluster, cloud platform, data sharing, task management
目录
摘要 I
Abstract II
第1章 绪论 1
1.1 研究背景 1
1.2 研究意义 1
1.3 研究内容 2
第2章 研究理论基础 4
2.1 Hadoop 4
2.2 MySQL 4
2.3 前端技术 5
2.4 Tomcat 5
2.5 Maven 6
2.6 高德API 6
2.7 其他工具 7
第3章 系统设计 8
3.1 总体设计 8
3.2 核心功能实现 9
3.2.1 数据共享功能 9
3.2.2 数据库连接 11
3.3 数据库设计 12
第4章 系统开发 16
4.1 数据共享功能 16
4.2.1 数据上传功能 16
4.2.2 数据检索功能 17
4.2.3 数据下载功能 18
4.2.4 数据删除功能 19
4.1 数据共享功能 19
4.2.1 任务发布与任务申请 19
4.2.1 任务自动调配 22
第5章 结论与展望 23
5.1 结论 23
5.2 展望 23
参考文献 25
致谢 26 |
