##
Bigdatais crucial formachinelearning, but the redundancies in the datasets are rarely studied. Here the authors reveal significantredundancyin large materials datasets, showing that up to 95 ...
Jul 29, 2024Themachinelearningpotential has emerged as a promising approach for addressing the accuracy-versus-efficiency dilemma in molecular modeling. Efficiently exploring chemical spaces with high accuracy presents a significant challenge, particularly for the interface reaction system. This study introduces a workflow aimed at achieving this goal by incorporating the classical SOAP descriptor and ...
In this paper, we improve the robustness ofMachineLearning(ML) classifiers againsttraining-time attacks by linking the risk oftrainingdatabeing tampered with to theredundancyin the ML model's design needed to prevent it.
Extensive efforts to gather materialsdatahave largely overlooked potentialdataredundancy. In this study, we present evidence of a significant degree ofredundancyacross multiple large datasets for various material properties, by revealing that up to 95 % ofdatacan be safely removed frommachinelearningtrainingwith little impact on in-distribution prediction performance. The redundant ...
Abstract Extensive efforts to gather materialsdatahave largely overlooked potentialdataredundancy. In this study, we present evidence of a significant degree ofredundancyacross multiple large datasets for various material properties, by revealing that up to 95% ofdatacan be safely removed frommachinelearningtrainingwith little impact on in-distribution prediction performance. The ...
Learning-based Methods: These methods learndataredundancyon somedatasamples (often sampled from thetrainingor validation datasets) and apply the knowledge at runtime execution of the model.
Extensive eforts to gather materialsdatahave largely overlooked potentialdataredundancy. In this study, we present evidence of a significant degree ofredundancyacross multiple large datasets for various material properties, by revealing that up to 95 % ofdatacan be safely removed fromma-chinelearningtrainingwith little impact on in-distribution prediction performance. The redundant ...
In this study, we present evidence of a significant degree ofredundancyacross multiple large datasets for various material properties, by revealing that up to 95 % ofdatacan be safely removed frommachinelearningtrainingwith little impact on in-distribution prediction performance.
Oct 18, 2024Thisredundancyskews the performance evaluation ofmachinelearning(ML) models when using random splitting, leading to overestimated predictive performance and poor performance on out-of ...
Jul 28, 2025Prerequisites: Chi-square test, covariance-and-correlation What isDataRedundancy? Duringdataintegration indatamining, variousdatastores are used. This can lead to the problem ofredundancyindata. An attribute (column or feature ofdataset) is called redundant if it can be derived from any other attribute or set of attributes. Inconsistencies in attribute or dimension naming can ...
Datanormalization organizesdata, reducesredundancy, and improves integrity. See types, examples, and methods for better analytics andmachinelearning.
Dec 17, 2025This paper presents a novel framework designed to significantly accelerate these pipelines. By establishing granulardataprovenance and implementing intelligent reuse strategies, our system efficiently identifies and eliminates redundant computations. This approach tackles key challenges such as managing extensivedatatraces and accommodating non-deterministic operations through advanced ...
Adatacenter is a facility used to house computer systems and associated components, such as telecommunications and storage systems. [1][2]Datacenters are critical infrastructure for the storage and processing of information, and they support the global financial system, cloud services,machinelearning, and artificial intelligence. [3][4]
2 days agoA central challenge in image-based Model-Based ReinforcementLearning(MBRL) is to learn representations that distill essential information from irrelevant visual details. While promising, reconstruction-based methods often waste capacity on large task-irrelevant regions. Decoder-free methods instead learn robust representations by leveragingDataAugmentation (DA), but reliance on such ...
1 day agoThis study proposes an interpretablemachine-learningpipeline for spectroscopicdatathat integrates PCA, supervised classification, and SHAP-based explainability.