Karma2: Provenance Management for Data-Driven Workflows

doi:10.4018/jwsr.2008040101

Reference Hub

This research has been cited in:

Conference
Visualization of network data provenance2012 19th International Conference on High Performance Computing10.1109/HiPC.2012.6507517
Article
GeoPWProv: Interleaving Map and Faceted Metadata for Provenance Visualization and NavigationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2013.2248064
Chapter
Capturing Provenance in the WildProvenance and Annotation of Data and Processes10.1007/978-3-642-17819-1_12
Article
Temporal representation for mining scientific data provenanceFuture Generation Computer Systems10.1016/j.future.2013.09.032
Article
Toward the modeling of data provenance in scientific publicationsComputer Standards & Interfaces10.1016/j.csi.2012.02.004
Chapter
A Mobile Cloud with Trusted Data Provenance Services for Bioinformatics ResearchData Provenance and Data Management in eScience10.1007/978-3-642-29931-5_5
Chapter
Towards Case-Based Support for e-Science Workflow Generation by Mining ProvenanceAdvances in Case-Based Reasoning10.1007/978-3-540-85502-6_18
Article
Explaining Natural Language query resultsThe VLDB Journal10.1007/s00778-019-00584-7
Article
Provenance Quality Assessment Methodology and FrameworkJournal of Data and Information Quality10.1145/2665069
Article
Provenance for Computational Tasks: A SurveyComputing in Science & Engineering10.1109/MCSE.2008.79
Article
A roadmap for privacy-enhanced secure data provenanceJournal of Intelligent Information Systems10.1007/s10844-014-0322-7
Chapter
Provenance Support for Data-Intensive Scientific WorkflowsGrid and Cloud Database Management10.1007/978-3-642-20045-8_11
Conference
A data lineage model for distributed sub-image processingProceedings of the 2010 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists10.1145/1899503.1899527
Chapter
Enabling Annotation Provenance in Bioinformatics Workflow ApplicationsAdvances in Bioinformatics and Computational Biology10.1007/978-3-642-15060-9_7
Article
Context provenance to enhance the dependability of ambient intelligence systemsPersonal and Ubiquitous Computing10.1007/s00779-011-0448-3
Conference
Achieving Linguistic Provenance via Plagiarism Detection2013 12th International Conference on Document Analysis and Recognition10.1109/ICDAR.2013.133
Article
A survey on provenance: What for? What form? What from?The VLDB Journal10.1007/s00778-017-0486-1
Chapter
A Noisy 10GB Provenance DatabaseBusiness Process Management Workshops10.1007/978-3-642-28115-0_35
Article
A graph distance based metric for data oriented workflow retrieval with variable time constraintsExpert Systems with Applications10.1016/j.eswa.2013.08.035
Article
A systematic view on data descriptors for the visual analysis of tabular dataInformation Visualization10.1177/1473871616667767
Conference
One Method for Provenance Tracking of Product Lifecycle Data in Collaborative Service Environment2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery10.1109/CyberC.2011.62
Article
Usage of Data Provenance Models in Collaborative Multidisciplinary Aero-Engine DesignJournal of Engineering for Gas Turbines and Power10.1115/1.4048436
Chapter
Tracking Files in the Kepler Provenance FrameworkScientific and Statistical Database Management10.1007/978-3-642-02279-1_21
Article
Tracing and using data lineage for pipeline processing in Astro-WISEExperimental Astronomy10.1007/s10686-011-9276-8
Chapter
Modelling Provenance Using Structured Occurrence NetworksProvenance and Annotation of Data and Processes10.1007/978-3-642-34222-6_14
Conference
OPQL: A First OPM-Level Query Language for Scientific Workflow Provenance2011 IEEE International Conference on Services Computing10.1109/SCC.2011.60
Article
Mechanisms for provenance collection in scientific workflow systemsComputing10.1007/s00607-017-0578-1
Article
A Dataflow-Based Scientific Workflow Composition FrameworkIEEE Transactions on Services Computing10.1109/TSC.2010.58
Conference
PLUS: A provenance manager for integrated information2011 IEEE International Conference on Information Reuse & Integration10.1109/IRI.2011.6009558
Conference
Provenance analysis: Towards quality provenance2012 IEEE 8th International Conference on E-Science10.1109/eScience.2012.6404480
Conference
Provenance-based object storage prediction scheme for scientific big data applications2014 IEEE International Conference on Big Data (Big Data)10.1109/BigData.2014.7004242
Article
Provenance Framework for Multi-Depth Querying Using Zero-Information Loss DatabaseInternational Journal of Information Technology & Decision Making10.1142/S0219622022500845
Chapter
A Systematic Literature Review on Numerical Weather Prediction Models and Provenance DataComputational Science and Its Applications – ICCSA 2022 Workshops10.1007/978-3-031-10542-5_42
Conference
Milieu: Lightweight and Configurable Big Data Provenance for Science2013 IEEE International Congress on Big Data10.1109/BigData.Congress.2013.16
Conference
A Characterization of the problem of secure provenance management2009 IEEE International Conference on Intelligence and Security Informatics10.1109/ISI.2009.5137332
Article
Nonintrusive collection and management of data provenance in scientific workflowsConcurrency and Computation: Practice and Experience10.1002/cpe.2809
Conference
Equivalence-Invariant Algebraic Provenance for Hyperplane Update QueriesProceedings of the 2020 ACM SIGMOD International Conference on Management of Data10.1145/3318464.3380578
Chapter
Fine-Grained Provenance Inference for a Large Processing Chain with Non-materialized Intermediate ViewsScientific and Statistical Database Management10.1007/978-3-642-31235-9_26
Chapter
Towards Automatic Capturing of Semi-structured Process ProvenanceData-Driven Process Discovery and Analysis10.1007/978-3-642-40919-6_5
Article
OPQL: Querying scientific workflow provenance at the graph levelData & Knowledge Engineering10.1016/j.datak.2013.08.008
Article
Selective provenance for datalog programs using top-k queriesProceedings of the VLDB Endowment10.14778/2824032.2824039
Article
Sub-image data processing in Astro-WISEExperimental Astronomy10.1007/s10686-012-9295-0
Article
NeuroBlocks – Visual Tracking of Segmentation and Proofreading for Large Connectomics ProjectsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2015.2467441
Article
ColtProceedings of the VLDB Endowment10.14778/3137765.3137783
Conference
AnnotationsProceedings of the 3rd workshop on Ph.D. students in information and knowledge management10.1145/1871902.1871904
Chapter
Provenance-Aware Faceted Search in DrupalProvenance and Annotation of Data and Processes10.1007/978-3-642-17819-1_17
Conference
An Approach to Standalone Provenance Systems for Big Social Provenance Data2016 12th International Conference on Semantics, Knowledge and Grids (SKG)10.1109/SKG.2016.010
Article
Secure abstraction views for scientific workflow provenance queryingIEEE Transactions on Services Computing10.1109/TSC.2010.38
Chapter
Looking Inside the Black-Box: Capturing Data Provenance Using Dynamic InstrumentationProvenance and Annotation of Data and Processes10.1007/978-3-319-16462-5_12
Article
Architecture design of a user-orientated electronic laboratory notebook: A case study within an atmospheric chemistry communityFuture Generation Computer Systems10.1016/j.future.2013.04.011
Conference
Provenance Information Model of Karma Version 32009 Congress on Services - I10.1109/SERVICES-I.2009.54
Conference
End-to-End Scientific Data Management Using Workflows2008 IEEE Congress on Services - Part I10.1109/SERVICES-1.2008.22
Conference
A process provenance service system for scientific data analysis and sharing2011 International Conference on Data and Knowledge Engineering (ICDKE)10.1109/ICDKE.2011.6053921
Conference
Temporal representation for scientific data provenance2012 IEEE 8th International Conference on E-Science10.1109/eScience.2012.6404477
Article
Provenance-based analysis of data-centric processesThe VLDB Journal10.1007/s00778-015-0390-5
Article
Towards a data‐driven IoT software architecture for smart city utilitiesSoftware: Practice and Experience10.1002/spe.2580
Article
CI-Miner: semantically enhancing scientific processesEarth Science Informatics10.1007/s12145-009-0038-3
Conference
Distributed Storage and Querying Techniques for a Semantic Web of Scientific Workflow Provenance2010 IEEE International Conference on Services Computing10.1109/SCC.2010.14
Article
An Inference-Based Framework to Manage Data Provenance in Geoscience ApplicationsIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2013.2247769
Conference
A User-orientated Electronic Laboratory Notebook for Retrieval and Extraction of Provenance Information for EUROCHAMP-22011 IEEE Seventh International Conference on eScience10.1109/eScience.2011.58
Conference
Provenance Collection in an Industry Biochemical Discovery Cyberinfrastructure2008 IEEE Fourth International Conference on eScience10.1109/eScience.2008.104
Chapter
Inferring Fine-Grained Data Provenance in Stream Data Processing: Reduced Storage Cost, High AccuracyDatabase and Expert Systems Applications10.1007/978-3-642-23091-2_11
Conference
Enactment engine independent provenance recording for e-science infrastructures2010 Fourth International Conference on Research Challenges in Information Science (RCIS)10.1109/RCIS.2010.5507287
Chapter
Provenance StorageEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_80746
Conference
Prospective and Retrospective Provenance Collection in Scientific Workflow Environments2010 IEEE International Conference on Services Computing10.1109/SCC.2010.18
Article
Everything you always wanted to know about a dataset: Studies in data summarisationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2019.10.004
Article
TITAN: A knowledge-based platform for Big Data workflow managementKnowledge-Based Systems10.1016/j.knosys.2021.107489
Article
High-throughput neuroimaging-genetics computational infrastructureFrontiers in Neuroinformatics10.3389/fninf.2014.00041
Chapter
Monitoring Unmanaged Business ProcessesOn the Move to Meaningful Internet Systems: OTM 201010.1007/978-3-642-16934-2_7
Article
A Comprehensive Query Language for Provenance InformationInternational Journal of Cooperative Information Systems10.1142/S0218843018500077
Article
Introducing Provenance Capture into a Legacy Data SystemIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2013.2282817
Article
Construction of a specialized integrated simulation platform for molecule screening based on scientific computing workflow engineScientific Reports10.1038/s41598-023-42913-5
Article
Sharing geospatial provenance in a service-oriented environmentComputers, Environment and Urban Systems10.1016/j.compenvurbsys.2011.02.006
Conference
Adaptive Inference of Fine-grained Data Provenance to Achieve High Accuracy at Lower Storage Costs2011 IEEE Seventh International Conference on eScience10.1109/eScience.2011.36
Chapter
Clouds and Reproducibility: A Way to Go to Scientific Experiments?Cloud Computing10.1007/978-3-319-54645-2_5
Article
Storing, reasoning, and querying OPM-compliant scientific workflow provenance using relational databasesFuture Generation Computer Systems10.1016/j.future.2010.10.013
Article
PROV2RACM Transactions on Internet Technology10.1145/3062176
Article
Applications of the pipeline environment for visual informatics and genomics computationsBMC Bioinformatics10.1186/1471-2105-12-304
Chapter
Provenance StorageEncyclopedia of Database Systems10.1007/978-1-4899-7993-3_80746-1
Article
The rationale of PROVJournal of Web Semantics10.1016/j.websem.2015.04.001
Article
Cost and accuracy aware scientific workflow retrieval based on distance measureInformation Sciences10.1016/j.ins.2015.03.055
Chapter
Probabilistic Inference of Fine-Grained Data ProvenanceDatabase and Expert Systems Applications10.1007/978-3-642-32600-4_22
Chapter
Managing Provenance in iRODSComputational Science – ICCS 200910.1007/978-3-642-01973-9_75
Conference
Hierarchical Caches for Grid Workflows2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid10.1109/CCGRID.2009.10
Chapter
Towards Next Generation Provenance Systems for E-ScienceFrameworks for Developing Efficient Information Systems10.4018/978-1-4666-4161-7.ch003
Article
Towards Next Generation Provenance Systems for e-ScienceInternational Journal of Information System Modeling and Design10.4018/jismd.2011070102
Chapter
Selective Service Provenance in the VRESCo RuntimeWeb Service Composition and New Frameworks in Designing Semantics10.4018/978-1-4666-1942-5.ch017

Karma2: Provenance Management for Data-Driven Workflows

Yogesh L. Simmhan, Beth Plale, Dennis Gannon

Source Title: International Journal of Web Services Research (IJWSR)5(2)

Cite Article Cite Article

MLA

Simmhan, Yogesh L., et al. "Karma2: Provenance Management for Data-Driven Workflows." IJWSR vol.5, no.2 2008: pp.1-22. http://doi.org/10.4018/jwsr.2008040101

APA

Simmhan, Y. L., Plale, B., & Gannon, D. (2008). Karma2: Provenance Management for Data-Driven Workflows. International Journal of Web Services Research (IJWSR), 5(2), 1-22. http://doi.org/10.4018/jwsr.2008040101

Chicago

Simmhan, Yogesh L., Beth Plale, and Dennis Gannon. "Karma2: Provenance Management for Data-Driven Workflows," International Journal of Web Services Research (IJWSR) 5, no.2: 1-22. http://doi.org/10.4018/jwsr.2008040101

Export Reference

Favorite Full-Issue Download

View Full Text PDF

Abstract

The increasing ability for the sciences to sense the world around us is resulting in a growing need for datadriven e-Science applications that are under the control of workflows composed of services on the Grid. The focus of our work is on provenance collection for these workflows that are necessary to validate the workflow and to determine quality of generated data products. The challenge we address is to record uniform and usable provenance metadata that meets the domain needs while minimizing the modification burden on the service authors and the performance overhead on the workflow engine and the services. The framework is based on generating discrete provenance activities during the lifecycle of a workflow execution that can be aggregated to form complex data and process provenance graphs that can span across workflows. The implementation uses a loosely coupled publish-subscribe architecture for propagating these activities, and the capabilities of the system satisfy the needs of detailed provenance collection. A performance evaluation of a prototype finds a minimal performance overhead (in the range of 1% for an eight-service workflow using 271 data products).

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.

Username or email: *

Password: *

Forgot individual login password?

Create individual account

Karma2: Provenance Management for Data-Driven Workflows

MLA

APA

Chicago

Export Reference

Abstract

Request Access