The disclosed invention includes an apparatus, system and method for developing tools to explore, organize, structure, extract, and mine natural language text. The system contains three sub-systems: a run-time engine, a development environment, and a feedback system. The invention also includes a system and method for improving the quality of information extraction applications consisting of an ensemble of per-user, adaptive, on-line machine-learning classifiers that adapt to document content and judgments of users by continuously incorporating feedback from information extraction results and corrections that users apply to these results. At least one of the machine-learning classifier also provides explanations or justifications for classification decisions in the form of rules; other machine-learning classifiers may provide feedback in the form of supporting instances or patterns.