Blockchain-Enabled Heuristic-Optimized Deduplication Model for Mitigating Single-Point Failures in Cloud Storage

Abstract

Cloud storage systems have become an essential component of modern data management, enabling users to store and access data remotely. However, traditional cloud storage architectures rely on centralized servers, which introduce critical challenges such as single-point failure, redundant data storage, high storage costs, and security vulnerabilities. In earlier systems, data was stored in centralized data centers where duplicate files were often saved multiple times, leading to inefficient utilization of storage resources. Although basic deduplication techniques were used, they frequently compromised data confidentiality and lacked transparency in metadata management. Moreover, failure of the central server could result in permanent data loss. To overcome these limitations, this research system integrates blockchain technology, InterPlanetary File System (IPFS), Convergent Encryption (CE), and heuristic-based chunking techniques to create a secure and decentralized storage framework, hereafter named Blockchain-enabled Heuristic Optimized Deduplication Model (BHODM). In this system, files are divided into optimized chunks using a heuristic method based on file size. Each chunk undergoes CE, where the encryption key is derived from the hash of the data itself, enabling secure deduplication without exposing plaintext information. Duplicate chunks are identified using hash comparison, ensuring that only unique data is stored. The encrypted chunks are stored in IPFS, a decentralized peer-to-peer storage network that eliminates reliance on a single server. Metadata such as file names, block numbers, and hash values are securely stored in an Ethereum blockchain smart contract, ensuring immutability and transparency. The system is implemented using Django for the web application, Web3 for blockchain interaction, IPFS Application Program Interface (API) for distributed storage, and Advanced Encryption Standard in Counter Mode (AES-CTR) encryption for security. By combining decentralized storage, blockchainbased metadata management, and secure deduplication, the proposed model effectively reduces storage overhead, enhances data integrity, and mitigates single-point failures. The system is further evaluated using storage utilization and computation time analysis, demonstrating improved efficiency compared to traditional approaches