Source-linked AI summary
A review on application of data mining techniques to combat natural disasters
Saptarsi Goswami, Sanjay Chakraborty, Sanhita Ghosh, Amlan Chakrabarti, Basabi Chakraborty
TL;DR
Natural disasters cause extensive loss of life and property, with India among the countries most affected by human-life losses. This paper reviews data-mining applications for prediction, detection, and disaster management, including data sources and a proposed India-focused system. It reports that social media and Internet sources contribute to early detection and disaster response.
Problem
Method
The paper reviews disaster data-mining techniques, task-specific data and sources, Twitter applications, and proposes a phased disaster-management database architecture for India.
Source: §1 · pp. 2–3, §7 · p. 14Results
The review finds that social media and Internet sources have contributed significantly to early disaster detection and appropriate response, while prediction uses varied data sources and methods.
Source: §4 · p. 10, §8 · p. 15Takeaways & Limitations
The review identifies social media as useful for real-time detection and understanding affected people’s needs, while proposing an India-focused system using Twitter and news feeds on Hadoop.
Source: §4 · p. 9, §7 · p. 14Twitter’s free Streaming API provides only a 1% sample, while full firehose access is prohibitively expensive; sampling parameters may need variation for truer coverage.
Source: §6 · p. 13
Abstract
from arXiv · showhide
Thousands of human lives are lost every year around the globe, apart from significant damage on property, animal life, etc., due to natural disasters (e.g., earthquake, flood, tsunami, hurricane and other storms, landslides, cloudburst, heat wave, forest fire). In this paper, we focus on reviewing the application of data mining and analytical techniques designed so far for (i) prediction, (ii) detection, and (iii) development of appropriate disaster management strategy based on the collected data from disasters. A detailed description of availability of data from geological observatories (seismological, hydrological), satellites, remote sensing and newer sources like social networking sites as twitter is presented. An extensive and in-depth literature study on current techniques for disaster prediction, detection and management has been done and the results are summarized according to various types of disasters. Finally a framework for building a disaster management database for India hosted on open source Big Data platform like Hadoop in a phased manner has been proposed. The study has special focus on India which ranks among top five counties in terms of absolute number of the loss of human life.
1. Introduction
Natural disasters cause widespread human, animal, and property losses, with India among the countries most affected by loss of human life. The paper reviews data-mining applications for disaster prediction, detection, and management, and proposes an India-focused disaster-management architecture.
1. Introduction
- India ranked among the top five countries for absolute human-life losses in three of the cited decades, while disasters also threaten health, businesses, and property.The introduction describes many data streams generated before, during, and after disasters, including satellite, sensor, government, NGO, and social-media data.Source: §1 · pp. 1–2
- The introduction describes earthquakes, landslides, cloudbursts, storms, floods, tsunamis, and volcanic eruptions as disaster types relevant to India.It notes that 55% of India's landmass is in seismic zones III–V, identifies the northern sub-Himalayan region and Western Ghats as landslide-prone, and describes cloudbursts as short-lived extreme rainfall.Source: §1 · p. 2
- The introduction also describes cyclones, floods, tsunamis, and volcanic eruptions, noting that underwater volcanoes and volcanoes on nearby islands had not significantly damaged the mainland.The cited examples include cyclone damage in coastal regions, annual flood losses, and the effects of the 2004 tsunami.Source: §1 · p. 2
- The paper’s contributions include a review of data-mining techniques and data sources across disaster tasks, an Indian disaster-management overview, a Twitter review, and a proposed system architecture.The paper is organized to cover task categories, task-level studies, data sources, Internet resources, and a proposed system.Source: §1 · pp. 2–3
2. NATURAL DISASTER FROM AN INDIAN CONTEXT
India faces multiple natural hazards and recurring losses, alongside reported weaknesses in disaster management. This section situates those challenges in the country’s geography and outlines the period covered by its disaster record.
2. NATURAL DISASTER FROM AN INDIAN CONTEXT
- India is described as experiencing about eight natural calamities per year, with roughly five times the frequency of natural disasters reported over the past three decades.The section reports 57% of landmass as earthquake-prone, 12% as flood-prone, and 8% as cyclone-prone; about 40 million hectares are vulnerable to floods.Source: §2 · p. 3
- Table 1 records natural disasters in India over the last 15 years.The supplied table passage identifies its scope but provides no event counts or other cell values.Source: Table 1 · p. 3, §2 · p. 3
- The section reports that Asia has the most disaster events among continents, with close to 60% of Asia’s events originating in South Asia and 40% in India.The cited passage says these statistics are displayed in Figure 1.Source: §2 · p. 3, Figure 1 · p. 3
- The section frames recurring disaster losses in India as a serious management problem and lists inadequate warnings, preparation, relief, administration, rehabilitation, and relief-finance management.It also identifies lack of effective help to victims among the reported shortcomings.Source: §2 · p. 4
National Disaster Management Structure
India’s disaster-management structure spans national institutions, ministries, state and district bodies, and local authorities. The section also names agencies assigned to floods and cyclones and ministries involved across the listed disasters.
National Disaster Management Structure
- The structure names national institutions and response bodies across state, district, and local levels, including NDMA, NIDM, response forces, police, fire services, and local authorities.It also lists cabinet committees, the Planning Commission, NEC, armed forces, paramilitary forces, Home Guard, NCC, and NSS.Source: National Disaster Management Structure · p. 4
- Flood responsibilities are assigned to the Ministry of Water Resources and CWC, while cyclones are assigned to the Indian Meteorological Department.Source: National Disaster Management Structure · p. 4
- The Ministry of Agriculture and Ministry of Animal Husbandry are involved in all the listed disaster areas.Source: National Disaster Management Structure · p. 5
3. BROAD CATEGORY OF TASKS WITH RESPECT TO NATURAL DISASTERS
The section groups disaster-related tasks into prediction, detection, and management strategies, while noting that these objectives can overlap. It also discusses psychological and behavioral study of affected regions.
3. BROAD CATEGORY OF TASKS WITH RESPECT TO NATURAL DISASTERS
- The task framework centers on prediction, detection, and disaster-management strategies, with classification based on each task’s direct objective.Source: §3 · p. 5
- Social-media reports can identify disasters faster than observatories, whose detected events may take time to reach authorities with an exact location.Source: §3 · p. 5
- Management tasks identify critical entities, communication needs, and affected-area needs to improve coordination and relief distribution; social-media data is important to this work.Source: §3 · p. 5
- Psychological and behavioral changes in affected regions are another study area, and prediction, detection, and these studies may overlap with management.The section classifies borderline cases by their direct task objective.Source: §3 · p. 5
- Prediction of a disaster itself is often beyond available data and techniques, but susceptible areas and characteristics such as earthquake magnitude or cyclone track and intensity can be predicted.Such tasks often use spatial and temporal data; a smaller research branch studies unusual animal behavior.Source: §3 · p. 5
- Disaster management aims to minimize casualties, rescue victims promptly, provide immediate first aid, evacuate people and animals, and reconstruct damage.Source: §3 · p. 5
4. DATA, MODELS, TASKS
The review organizes disaster-related research by disaster type, task, model, data, and study region, then highlights patterns in the literature. It identifies prediction and disaster management as the main research tasks and notes limited Indian research despite the country's substantial disaster losses.
4. DATA, MODELS, TASKS
- Tables 2a–2h organize granular tasks, models, data, and study regions by disaster, with a final table for generic efforts.The survey describes its coverage as an effort to review roughly the last 10–12 years, not an exhaustive list.Source: §4 · pp. 5, 9
- 10 of the 40 reviewed papers use Twitter, but the authors found no referential work using it in an Indian context and report less Indian research despite greater losses.The country-of-analysis figure is cited as supporting the comparison.Source: §4 · p. 9, Figure 3 · p. 9
- The review estimates research activity by disaster type using Google Scholar results filtered to since 2014, without limiting searches to data-mining applications.Figure 4 presents the search-result counts.Source: §4 · p. 9, Figure 4 · p. 10
- Prediction and disaster management are the main research tasks; other efforts build data warehouses and OLAP structures for information dissemination and consumption.The review summarizes prediction tasks by disaster type in Table 3.Source: §4 · p. 10, Table 3 · p. 10
5. DATA SOURCES &TYPES
The section surveys data types, formats, sources, and access relevant to disaster analysis, including volume, variety, and velocity dimensions. It also notes a threshold-triggered wireless sensor approach for flash-flood alerts and the limits of numerical weather prediction for direct event prediction.
5. DATA SOURCES &TYPES
- Disaster-related data span volume, variety, and velocity, including GIS, meteorological, social-media, text, time-series, spatial, and image data.The section frames its data-source inventory around types, formats, collecting agencies, access, and availability.Source: §5 · p. 11, Table 4 · p. 11
- A proposed flash-flood setup uses a water-level sensor that broadcasts a danger signal when a threshold is reached; deployment was planned for areas of Delhi.The sensor sends its signal to a receiver station.Source: §5 · p. 12, Figure 6 · p. 12
- Numerical weather prediction models solve atmospheric equations for short-term forecasting, but cannot directly predict actual events; statistical techniques are also needed.The passage says most meteorological departments have adopted NWP models.Source: §5 · p. 12
6. TWITTER AND SOCIAL MEDIA AS SOURCES
The section presents Twitter and other online sources as inputs for disaster analysis, while describing their uses and data-access constraints. It notes that Twitter is used for faster detection, relief-organization identification, and tracking needs, but that Indian-context references were not found in the reviewed work.
6. TWITTER AND SOCIAL MEDIA AS SOURCES
- Tweets offer short text, hashtags, and geocodes, while available fields include sender identifiers, language codes, coordinates, and timestamps.News channels and blogs can also be accessed through APIs, RSS feeds, or web scraping.Source: §6 · p. 13
- Twitter has been used to detect disasters faster than observatories, identify relief entities, and study needs and concerns over time,,,,,,,,,.The review notes little referential work in the Indian context.Source: §6 · p. 13
- Twitter's free Streaming API provides only a 1% sample, while full Firehose access is prohibitively expensive; the review says varied sampling parameters are needed for a truer picture.The cited comparison found some agreement between the two extraction approaches.Source: §6 · p. 13
7. PROPOSED SYSTEM
The proposed system starts with Internet-based disaster data, especially Twitter and news, and plans to expand to other sources in a later phase. Its Hadoop-based architecture covers collection, processing, storage, analysis, and reporting.
7. PROPOSED SYSTEM
- Phase 1 collects disaster information from Twitter, news, other social media, and Internet sources; Phase 2 adds data sources beyond the Internet.The proposal aims to build a database for natural disasters in India.Source: §7 · pp. 13–14
- The process flow includes streaming/API collection and web scraping, followed by text cleanup, hashtag extraction, news deduplication, feature selection, topic clustering, and geographic information.The proposal identifies Figure 7 as the system process flow.Source: Figure 7
- The proposed system plans to store data in a distributed system like Hive, create separate disaster subject areas, use exploratory techniques to find trends and relief analysis, and disseminate information through charts, reports, or dashboards.Figure 8 is labeled as the high-level architecture of the proposed system.Source: Figure 8 · p. 14
- Unlike cited systems focused on Twitter or a geo-hazards inventory, the proposal targets wider sources and understanding affected people’s needs during disasters.The proposal also mentions evaluating social information, but the supplied passage does not specify how.Source: §7 · p. 14
- The proposed architecture uses Twitter and news RSS feeds, with scripts in Flume, Scoop, or R for extraction, and Hadoop for storage and processing.Data may reside in HDFS or Hive after processing with Pig; planned algorithms use Mahout.Source: §7 · p. 14, Figure 8 · p. 14
8. CONCLUSION
The review finds that social media and Internet sources contribute to early disaster detection and response, while work using these sources remains limited in India. It proposes a phased Indian disaster data store, beginning with these sources and later integrating others.
8. CONCLUSION
- The review concludes that social media and Internet sources have contributed to early disaster detection and appropriate response, but their potential remains insufficiently explored in India.The conclusion distinguishes these sources’ role in detection and response from disaster prediction.Source: §8 · p. 15
- The paper synthesizes data-mining applications across disaster prediction, detection, and management, emphasizing the multidisciplinary nature of the tasks and their varied data sources.The conclusion notes that these applications require subject-matter expertise.Source: §8 · p. 15
- The authors propose an Indian natural-disaster data store using social media and Internet sources in Phase 1, with other information sources to be integrated in Phase II.Source: §8 · p. 15