Implementing Splunk: Big Data Essentials for Operational Intelligence ppt

448 2,472 1
  • Loading ...
    Loading ...
    Loading ...

Tài liệu hạn chế xem trước, để xem đầy đủ mời bạn chọn Tải xuống

Tài liệu liên quan

Thông tin tài liệu

Ngày đăng: 07/03/2014, 04:20 Splunk: Big Data Reporting and Development for Operational IntelligenceLearn to transform your machine data into valuable IT and business insights with this comprehensive and practical tutorialVincent BumgarnerBIRMINGHAM - Splunk: Big Data Reporting and Development for Operational IntelligenceCopyright © 2013 Packt PublishingAll rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.First published: January 2013Production Reference: 1140113Published by Packt Publishing Ltd.Livery Place35 Livery StreetBirmingham B3 2PB, UK.ISBN 978-1-84969-328-8www.packtpub.comCover Image by Vincent Bumgarner ( BumgarnerReviewersMathieu DessusCindy McCririeNick MealyAcquisition EditorKartikey PandeyLead Technical EditorAzharuddin SheikhTechnical EditorsCharmaine PereiraVarun Pius RodriguesCopy EditorsBrandt D'MelloAditya NairAlda PaivaLaxmi SubramanianRuta WaghmareProject CoordinatorAnish RamchandaniProofreaderMartin DiverIndexerTejal SoniGraphicsAditi GajjarProduction CoordinatorNitesh ThakurCover WorkNitesh the AuthorVincent Bumgarner has been designing software for nearly 20 years, working in many languages on nearly as many platforms. He started using Splunk in 2007 and has enjoyed watching the product evolve over the years.While working for Splunk, he helped many companies, training dozens of users to drive, extend, and administer this extremely exible product. At least one person at every company he worked with asked for a book on Splunk, and he hopes his effort helps ll their shelves.I would like to thank my wife and kids as this book could not have happened without their support. A big thank you to all of the reviewers for contributing their time and expertise, and special thanks to SplunkNinja for the the ReviewersMathieu Dessus is a security consultant for Verizon in France and acts as the SIEM leader for EMEA. With more than 12 years of experience in the security area, he has acquired a deep technical background in the management, design, assessment, and systems integration of information security technologies. He specializes in web security, Unix, SIEM, and security architecture design.Cindy McCririe is a client architect at Splunk. In this role, she has worked with several of Splunk's enterprise customers, ensuring successful deployment of the technology. Many of these customers are using Splunk in unique ways. Sample use cases include PCI compliance, security, operations management, business intelligence, Dev/Ops, and transaction proling.Nick Mealy was an early employee at Splunk and worked as the Mad Scientist / Principal User Interface Developer at Splunk from March 2005 to September 2010. He led the technical design and development of the systems that power Splunk's search and reporting interfaces as well as on the general systems that power Splunk's congurable views and dashboards. In 2010, he left Splunk to found his current company, Sideview, which is creating new Splunk apps and new products on top of the Splunk platform. The most widely known of these products is the Sideview Utils app, which has become very widely deployed (and will be discussed in Chapter 8, Building Advanced Dashboards). Sideview Utils provides new UI modules and new techniques that make it easier for Splunk app developers and dashboard creators to create and maintain their custom views and les, eBooks, discount offers and moreYou might want to visit for support les and downloads related to your book. Did you know that Packt offers eBook versions of every book published, with PDF and ePub les available? You can upgrade to the eBook version at and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.At, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books. Why Subscribe?• Fully searchable across every book published by Packt• Copy and paste, print and bookmark content• On demand and accessible via web browserFree Access for Packt account holdersIf you have an account with Packt at, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate of ContentsPreface 1Chapter 1: The Splunk Interface 7Logging in to Splunk 7The Home app 8The top bar 11Search app 13Data generator 13The Summary view 14Search 16Actions 17Timeline 18The eld picker 19Fields 19Search results 21Options 22Events viewer 23Using the time picker 25Using the eld picker 26Using Manager 27Summary 29Chapter 2: Understanding Search 31Using search terms effectively 31Boolean and grouping operators 32Clicking to modify your search 34Event segmentation 34Field widgets 34Time of Contents[ ii ]Using elds to search 35Using the eld picker 35Using wildcards efciently 36Only trailing wildcards are efcient 36Wildcards are tested last 36Supplementing wildcards in elds 37All about time 37How Splunk parses time 37How Splunk stores time 37How Splunk displays time 38How time zones are determined and why it matters 38Different ways to search against time 39Specifying time in-line in your search 41_indextime versus _time 42Making searches faster 42Sharing results with others 43Saving searches for reuse 46Creating alerts from searches 48Schedule 49Actions 51Summary 52Chapter 3: Tables, Charts, and Fields 53About the pipe symbol 53Using top to show common eld values 54Controlling the output of top 56Using stats to aggregate values 57Using chart to turn data 61Using timechart to show values over time 63timechart options 65Working with elds 66A regular expression primer 66Commands that create elds 68eval 68rex 69Extracting loglevel 70Using the Extract Fields interface 70Using rex to prototype a eld 73Using the admin interface to build a eld 75Indexed elds versus extracted elds 77Summary of Contents[ iii ]Chapter 4: Simple XML Dashboards 81The purpose of dashboards 81Using wizards to build dashboards 82Scheduling the generation of dashboards 91Editing the XML directly 91UI Examples app 92Building forms 92Creating a form from a dashboard 92Driving multiple panels from one form 97Post-processing search results 104Post-processing limitations 106Panel 1 106Panel 2 107Panel 3 108Final XML 108Summary 110Chapter 5: Advanced Search Examples 111Using subsearches to nd loosely related events 111Subsearch 111Subsearch caveats 112Nested subsearches 113Using transaction 114Using transaction to determine the session length 115Calculating the aggregate of transaction statistics 117Combining subsearches with transaction 118Determining concurrency 122Using transaction with concurrency 122Using concurrency to estimate server load 123Calculating concurrency with a by clause 124Calculating events per slice of time 129Using timechart 129Calculating average requests per minute 131Calculating average events per minute, per hour 132Rebuilding top 134Summary 141Chapter 6: Extending Search 143Using tags to simplify search 143Using event types to categorize results 146Using lookups to enrich data 150Dening a lookup table le[...]... Splunk instance types Splunk forwarders Splunk indexer Splunk search Common data sources Monitoring logs on servers Monitoring logs on a shared drive Consuming logs in batch Receiving syslog events Receiving events directly on the Splunk indexer Using a native syslog receiver Receiving syslog with a Splunk forwarder Consuming logs from a database Using scripts to gather data Sizing indexers Planning... Writing commands When not to write a command When to write a command Configuring commands Adding fields Manipulating data [ viii ] 379 379 380 382 384 385 387 390 390 392 392 393 394 Table of Contents Transforming data Generating data Writing a scripted lookup to enrich data Writing an event renderer Using specific fields Table of fields based on field value Pretty print XML Writing... extend Splunk for data input, external querying, rendering, custom commands, and custom actions What you need for this book To work through the examples in this book, you will need an installation of Splunk, preferably a non-production instance If you are already working with Splunk, then the concepts introduced by the examples should be applicable to your own data Splunk can be downloaded for free from... try to act as a replacement for the official Splunk documentation, but should serve as a shortcut for many concepts For some sections, a good understanding of regular expressions would be helpful For some sections, the ability to read Python would be helpful Conventions In this book, you will find a number of styles of text that distinguish between different kinds of information Here are some examples... to Splunk We will discuss apps in a more detailed manner in Chapter 7, Working with Apps Under Do more with Splunk, we find: • Add data: This links to the Add Data to Splunk page This interface is a great start for getting local data flowing into Splunk The new Preview data interface takes an enormous amount of complexity out of configuring dates and line breaking We won't go through those interfaces... For instance, if the system is configured to use Windows Active Directory via LDAP (a very common configuration), users must change their password in Windows Search app The search app is where most actions in Splunk start Data generator If you want to follow the examples that appear in the next few chapters, install the ImplementingSplunkDataGenerator demo app by following these steps: 1 Download ImplementingSplunkDataGenerator.tar.gz... indexed data panel shows statistics for all indexed data Remember that this only reflects indexes that this particular user searches by default There are other events that are indexed by Splunk, including events Splunk indexes about itself We will discuss indexes in Chapter 9, Building Advanced Dashboards The next three panels give a breakdown of your data using three important pieces of metadata—source,... be thousands of machines submitting data, but all data at the same path across these machines counts as one source When the data source is not a file, the value of the source can be arbitrary, for instance the name of a script or network port A source type is an arbitrary categorization of events There may be many sources across many hosts in the same source type For instance, given the sources /var/... which provides very detailed information about the query that was run • Print ( ), which formats the page for printing and instructs the browser to print [ 17 ] The Splunk Interface • Save, which provides different options for saving the search or the results We will discuss this later in this chapter • Create, which provides wizard-like interfaces for building different objects..., for most popular platforms The sample code was developed on a Unix system, so you will probably have better luck using an installation of Splunk that is running on a Unix operating system Knowledge of Python is necessary to follow some of the examples in the later chapters [2] Preface Who this book is for This book should be useful for new users, seasoned . Implementing Splunk: Big Data Reporting and Development for Operational Intelligence Learn to transform your machine data into valuable. BumgarnerBIRMINGHAM - Implementing Splunk: Big Data Reporting and Development for Operational Intelligence Copyright © 2013 Packt PublishingAll
- Xem thêm -

Xem thêm: Implementing Splunk: Big Data Essentials for Operational Intelligence ppt, Implementing Splunk: Big Data Essentials for Operational Intelligence ppt, Implementing Splunk: Big Data Essentials for Operational Intelligence ppt, Step 5 – Mapping these apps to deployment clients in serverclass.conf

Mục lục

Xem thêm