Risk Mgmt Unit 3: Predicting Risk: Approaches using Artificial Intelligence and Machine Learning

Upon successful completion of this unit, learners will be able to identify how to use artificial intelligence and machine learning to predict levels and types of risk, both known and unknown.  Links to open source platforms, languages, and computing environments are provided.  It is not necessary to learn the computing languages or to develop new code or programs; the goal of this unit is to familiarize learners in order to work effectively in teams with data scientists, domain experts, and financial decision-makers.

Unit Presentation:

Video:   https://screencast-o-matic.com/watch/cqVZht3OAh

PDF (contains links to readings, etc.) 
http://zenzebra.net/risk/risk-management-nash-pt3.pdf  

Scenario 3:  Predicting Risk: Approaches using Artificial Intelligence and Machine Learning

Julia, Patricio, and Reyna are part of a team that is tasked with classifying old shallow-water offshore wells in the Gulf of Mexico in new ways that will help them develop a plan to boost production. 

They feel very fortunate in that around a million geological and production records have been scanned, and they cover the 150 or so wells in the field.  It’s a treasure trove of data, and they want to incorporate it with the new data in order to develop a profile of the best wells, as well as the good, mediocre, and underperforming wells.

Your Task: Help Julia, Patricio, and Reyna develop a plan to analyze the data, and then help them determine where, when, and how they can use artificial intelligence and machine learning to create profiles.

Here are a few things to consider:
 How will you select the data to use?
 How will you organize it?

What does it mean for a well to be:
  Excellent
  Good
  Mediocre
  Bad
 
What are the attributes or clusters of characteristics you’ll use?
 What approach will you use to select data?
  To clean the data?
  To analyze the data?
 What kind of AI / ML approach will you use?
 How will you use the results?

Readings:
Overview thoughts / concepts

Lists of uses of AI / Machine Learning the energy industry
 Upstream
  Classify wells using your own unique set of criteria
  Identify high-value (or potential high-value) blocks
 Midstream
  Classify infrastructure (pipelines, etc) with your own criteria
  Predict overall performance and the location of bottlenecks
 Downstream
  Refining
  Retail / distribution
 Wind energy  Identify high-value, high-return new locations
  Identify small businesses that would benefit from local energy
 Solar energy
Workflow for machine learning (in general)

● Pinpoint the problem you want to solve.
● Identify the data you’ll need to use
● Collect the data
● Clean the data
● Organize your data (put into a model - if structured, may use Open Source models such as those from Apache HaDoop)
● Find a model
● Develop algorithms (May use repositories and also cloud-based interfaces)
● Train the model
● Test with data sets
● Reality check
● Decision points
 
How do I clean data?
 What is “dirty” data? 
  Does not make sense
  Bad labels
  Incorrect formatting
  Too many “nulls”
  Part of the data in a different order or different columns
Brendon Bailey’s Guide:  Use Excel or Python to Clean Data?

Use Excel if: You have fewer than 1 million records
You need to do the job quick and easy
There is a logical pattern to cleaning the data and it’s easy enough to clean using Excel functions
The logical pattern to cleaning the data is hard to define, and you need to clean the data manually
When you might use Python or another scripting language:

Use Python if: You need to document your process
You plan on doing the job on a repeat basis
There is a logical pattern to cleaning the data, but it is hard to implement with Excel functions

Brendon Bailey. “Data Cleaning 101” TowardDataScience.com
 https://towardsdatascience.com/data-cleaning-101-948d22a92e4
 
Where do you keep the data?
 cloud solutions (Google, Amazon Web Services (AWS))

Software for risk analytics (free / open source):

Spotfire (http://www.spotfire.com)
Qlik.com (free Spotfire alternative, Qlik.com)
Jupyter Notebook https://jupyter.org/
 iPython
 R
 C++
 Julia

A Gallery of interesting Jupyter Notebooks (ready to share)
https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks

How do we predict where and when high-risk situations may take place?
 Analyze data
 Probabilistic analysis (Spotfire, etc.)
 Using geospatial elements
What is the ideal combination of variable or factors to tell us when / where / how conditions are ideal for a) optimization; b) an accident or problem ?
 Use multivariate analysis
 Bring together all risk factors: geological, logistical, political, economic, legal, environmental, etc.
 Weight them by importance (assign a percentage)

https://www.kinetica.com/wp-content/uploads/2017/09/OilGas_jt1.0mn.pdf https://medium.com/syncedreview/how-ai-can-help-the-oil-industry-b853dda86be6

Learn and Use Machine Learning

Tensorflow: https://www.tensorflow.org/tutorials/keras/

Tensorflow Machine Learning Cookbook: https://github.com/nfmcclure/tensorflow_cookbook

AI and Probabilistic Models

Part I
https://medium.com/tensorflow/industrial-ai-bhges-physics-based-probabilistic-deep-learning-using-tensorflow-probability-5f215c791863

Part II
https://medium.com/tensorflow/predicting-known-unknowns-with-tensorflow-probability-industrial-ai-part-2-2fbd3522ebda

Bougher, Benjamin Bryan. (2016)  Machine Learning Applications to Geophysical Data Analysis. Open Collections. University of British Columbia.
https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0308786

Bougher, Ben B. (2016) Using the scattering transform to predict stratigraphic units from well logs. Seismic Laboratory for Imaging and Modeling (SLIM), The University of British Columbia, Vancouver

https://www.slim.eos.ubc.ca/Publications/Public/Journals/CSEGRecorder/2016/bougher2015CSEGust/bougher2015CSEGust.html

Data:  Trenton Black River gamma ray logs

Methodology:  supervised learning ("uses labelled datasets to train a classifier to make predictions about future data" (Bougher, 2016))

Methodology - what's the algorithm?  Bougher uses a scattering transform - and then it fieeds a K-Nearest Neighbours (KNN) classifier).

How can I do this?

Using convolutional neural networks to solve a mineral prospectivity mapping problem
Framing the exploration task as a supervised learning problem, the geological, geochemical and geophysical information can be used as training data, and known mineral occurrences can be used as training labels. The goal is to parameterize the complex relationships between the data and the labels such that mineral potential can be estimated in under-explored regions using available geoscience data.

Granek, Justin. (2016). Application of Machine Learning Algorithms to Mineral Prospectivity Mapping. Open Collections. University of British Columbia.
https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0340340


TAMUT  MBA in Energy Leadership: Click link to apply - more information

For more information about the courses (and this full course), please contact me. 

OpenSource Webinar Software: Updated List of Platforms and Software/Plugins

Are you outgrowing your webinar capacity? Do you need more flexibility, lower cost, and more control? Open source may be a great solution for you, particularly as solutions evolve and popular opensource learning solutions such as Moodle offer convenient and relatively easy-to-implement plug-ins. There are open source solutions for many organizational needs (college, university, association, business, school), and there are also services that will host for you.

One of E-Learning Queen's most popular posts in the past detailed open source webinar solutions. You may still visit that post. Times change quickly, though, and many of the solutions listed there are no longer viable.

So, as a service to readers, E-Learning Queen is publishing an updated list. Please feel free to share your experiences with the solutions. We'd love to hear your thoughts.

Moodle Solutions

Moodle Plugin from Sclipo

http://sclipo.com/plugins/moodle

Download from SourceForge

http://sourceforge.net/projects/moodlewebclass/

Step-by-step Install Guide Moodle on Linux with Sclipo Live Web Class on Linux v1.4

http://www.scribd.com/doc/17658143/Stepbystep-Install-Guide-Moodle-on-Linux-with-Sclipo-Live-Web-Class-on-Linux-v14

Free PHP Code: http://phpkode.com/source/p/moodle-live-classes-and-webinar-plugin/sclipowebclass/index.php

Video Tutorials: How to install Sclipo

http://wn.com/sclipo

Bundled Open Source Solutions

BigBlueButton

http://bigbluebutton.org/

Uses all open-source components (http://bigbluebutton.org/components)

Colloquium

http://sourceforge.net/projects/colloquium/

Colloquium is a pedagogical tool for presenting online webcasts with supplementary materials. Streaming video is combined with a rich set of support materials - live chat, threaded discussion, and links.

Webcasting Plus

DCAST

http://sourceforge.net/projects/dcast/

EasyCast

http://easycast.sourceforge.net/

Easy in use system of scripts to organize webcast from conferences and other such events. Allows maintaining video stream and presentation viewing together with asking questions from internet.

Desktop Sharing

Apreta

http://sourceforge.net/projects/apreta/

Apreta is about sharing information in real-time meetings. It provides desktop sharing, presentation sharing, a shared whiteboard, and audio conferencing.

Desktop Share - Class Room Integration (CRI)

http://cri.sourceforge.net/

This is a web application that enables integration between the teacher and student in a virtual classroom. The application gives users the ability to share a desktop with Teacher/Coordinator/Admin and Student can share his her desktop with Teacher/Coordinator/Admin.

Wedatoo

http://sourceforge.net/projects/wedatoo/

Wedatoo is an open source point to point realtime collaboration application like google talk which features text chat, file transfers, audio/video conferencing, whiteboard, image sharing and desktop sharing/control modeled on google talk.

DimDim

http://sourceforge.net/projects/dimdim/

http://www.dimdim.com

DimDim has an opensource webinar solution which is, at this point, free. DimDim has been acquired by Salesforce.com. We will see what happens.

Webmeeting

http://sourceforge.net/projects/webmeeting/

Web Conferencing system allow you to conduct the web conference around the globe. The system allow to share the desktop , application, dashboard, voice chatting and video conferencing.

ILIAS LMS

http://www.ilias.de/docu/

A full-featured opensource learning management system. Not apparent whether or not it has a plugin that allows webinars / webcasting with chat / whiteboard.

Webcasting

UStream

http://www.ustream.tv

Streaming and webcasting do not have the interactivity of a packaged solution of WebEx, Omnovia, or GoToMeeting. However, you can piggyback services, and use UStream in conjunction with messaging / doc sharing

DVLabs

http://www.dvlabs.com/services.php

Stripped-down basic functionality?

Sofia-SIP

http://sofia-sip.sourceforge.net/

Session Initiation Protocol software

Moodle Security: A Review

Even if you're using a third-party service for your learning management system (LMS), it is still important to be mindful of security, and to understand the ins and outs of where the vulnerabilities exist. For Moodle administrators, the issue of security is important on a number of levels. First, are privacy issues and regulations; second, are integrity issues, as you seek to keep your information and data intact; third, are authentication issues to protect users. Finally, is the issue of ongoing bot activity that can potentially hijack your server and make it perform in ways that you would prefer it not to.

Packt Publishing's Moodle Security (https://www.packtpub.com/toc/moodle-security-table-contents) addresses concerns of both administrators and users. The book provides information that can be helpful in addressing the very real legal liabilities that occur with security breaches.



The organization of the book is easy to follow, and screen shots allow the user to work through the processes in a step-by-step way, and it allows both Linux and Windows users to implement security protocols for Moodle.

The first several chapters have to do with securing Linux and Windows servers. The chapters that deal with authentication, roles, and permissions include a variety of configurations, that include installation procedures.

On the side of user authentication and content assurance, the chapters are thorough and clear. Some current and ever-evolving threats -- bots, viruses, and other types of vulnerabilities are dealt with on a conceptual level as well as with step-by-step instructions.

One area that could be expanded in the Moodle Security is the underlying approach to computer forensics and historical approaches to determining crime / criminal elements.

Note: This title and other Moodle titles are available at a discount through the publisher for a limited time. (Moodle March campaign).

Creating Webinars Using Open Source Software

Do you want to hold a webinar, but want to save money by doing it on your own? An updated list of sources can be found here. As speed and bandwidth restrictions continue to be reduced, and travel costs and time away from the office continue to be problematic, webinars are being incorporated into education, professional development, training, and team-building.

Don't Miss the Updated List of Open-Source Webinar Software and Platforms!

http://elearnqueen.blogspot.com/2011/05/opensource-webinar-software-updated.html

Open Source software can be a great solution when there is sufficient programming and network support expertise to be able to deal with some of the challenges of beta-level (or even alpha) software. However, the rewards of open source can be significant for those wishing to maintain control, develop a brand image, reduce monthly operating costs, and to even offer hosting / repackaged solutions for a specialized niche market.

For colleges and universities, the webinar format can be a wonderful way to present guest lecturers, and to discuss research design, particularly for sociology, psychology, and earth sciences. For corporations and professional associations, the webinar format can be effective. It is important, however, to plan well and to avoid common pitfalls.

Assemble your own "rough and ready" webinar?
It could be possible to simply broadcast a live event using ustream (http://www.ustream.tv/) while simultaneously using MSN or AIM for chat and discussion capabilities. This could result in a haphazard event, with archives of the chat avaialable in a different place than the archived video. However, it's a "rough and ready" way to do a webinar if you want to stay free, and not have to progress to a for-pay version of gotomeeting, skype, webex, omnovia, elluminate, and other providers of web conferencing and synchronous web training.

OpenSource Webinar Software
(please note that these products have been reviewed, but not installed and tested by e-learning queen).

vmukti
http://www.vmukti.com/
VMukti is an open-source, cloud-based high definition video conferencing communication suite, which has three main features:
Live Video Streaming and broadcasting
Web Conferencing and Collaboration
Video Voip Telephony
In theory, VMukti offers live video streaming in an affordable, effective manner. Total video product on cost, Quality and Scalability. Server less, Client Less, Low cost, High Definition, Scalable, Failover.

Webhuddle
http://www.webhuddle.com/
-Small footprint
-Easy to use: client runs in web browser (no installation)
-Thin Client
-Secure
-Offers hosting services

OpenMeetings
OpenSource
http://code.google.com/p/openmeetings/
OpenMeetings is a free browser-based software that allows you to set up instantly a conference in the Web. You can use your microphone or webcam, share documents on a white board, share your screen or record meetings. It is available as hosted service or you download and install a package on your server with no limitations in usage or users.

BigBlueButton
BigBlueButton is an open source web conferencing system for distance education. It provides real-time desktop sharing, presentation, VoIP, webcam and chat.

WebConference Plugin for Moodle
Plug in WebConference Internet conferencing into your Moodle Learning Management System. This mod for Moodle 1.9.7+ will enable your moodle installation to add WebConference Activities to courses. Adjust seats, and schedule conference events.

The Ehizo Project --- Communication Suite
data & video-conferencing, one shared whiteboard, send text messages (IM), transfer files(FTP),collaborate in real time,email capablities,Plays and stores dvd,cd and Mp3. Web-based Internet directory,Remote Desktop Share,File sharing program & browser

1videoConference
http://1videoconference.com/
(modified from website) An Open Source web conferencing company, 1VideoConference is allows its Web, Audio/ Video phone, Skype, Msn and Yahoo users to instantly participate in live web conferences without the need for lengthy downloads or complicated installations. At any time, our host customers can simply drop a small piece of code onto their websites and create an online video conference room.

The beta version of 1videoConference is available at sourceforge.net for download and installation on one's own server. This is an opensource alpha version with known bugs and little aesthetic design.

*****
From the archives of fringejournal.com: "Feed the Bears Anyway"