jupyter notebooks for transmogrify samples by rajdeepd · Pull Request #231 · salesforce/TransmogrifAI

rajdeepd · 2019-02-24T15:16:11Z

Related issues
Refer to issue(s) addressed in this pull request from [Issues]

#211

Describe the proposed solution
Jupyter notebook samples based on BeakerX based kernel

Describe alternatives you've considered

Apache Toree, almond-sh based kernels, most of these solutions were not as straight forward as beakerx

Additional context
none

codecov · 2019-02-24T15:34:57Z

Codecov Report

Merging #231 into master will decrease coverage by 4.01%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #231      +/-   ##
=========================================
- Coverage   86.42%   82.4%   -4.02%     
=========================================
  Files         312     312              
  Lines       10187   10187              
  Branches      336     548     +212     
=========================================
- Hits         8804    8395     -409     
- Misses       1383    1792     +409

Impacted Files	Coverage Δ
...alesforce/op/cli/gen/templates/SimpleProject.scala	`0% <0%> (-100%)`	⬇️
.../scala/com/salesforce/op/cli/gen/ProblemKind.scala	`0% <0%> (-100%)`	⬇️
...cala/com/salesforce/op/cli/gen/FileInProject.scala	`0% <0%> (-100%)`	⬇️
...in/scala/com/salesforce/op/cli/CommandParser.scala	`0% <0%> (-98.12%)`	⬇️
...cala/com/salesforce/op/cli/gen/ProblemSchema.scala	`0% <0%> (-96.56%)`	⬇️
...src/main/scala/com/salesforce/op/cli/gen/Ops.scala	`0% <0%> (-94%)`	⬇️
...ain/scala/com/salesforce/op/cli/SchemaSource.scala	`0% <0%> (-87.94%)`	⬇️
...a/com/salesforce/op/cli/gen/ProjectGenerator.scala	`0% <0%> (-87.5%)`	⬇️
...in/scala/com/salesforce/op/cli/CliParameters.scala	`0% <0%> (-80.96%)`	⬇️
...src/main/scala/com/salesforce/op/cli/CliExec.scala	`0% <0%> (-80%)`	⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4da94f8...f786cb9. Read the comment docs.

codecov · 2019-02-24T15:34:58Z

Codecov Report

Merging #231 into master will decrease coverage by 4%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #231      +/-   ##
==========================================
- Coverage   86.39%   82.39%   -4.01%     
==========================================
  Files         312      312              
  Lines       10183    10183              
  Branches      335      548     +213     
==========================================
- Hits         8798     8390     -408     
- Misses       1385     1793     +408

Impacted Files	Coverage Δ
...alesforce/op/cli/gen/templates/SimpleProject.scala	`0% <0%> (-100%)`	⬇️
.../scala/com/salesforce/op/cli/gen/ProblemKind.scala	`0% <0%> (-100%)`	⬇️
...cala/com/salesforce/op/cli/gen/FileInProject.scala	`0% <0%> (-100%)`	⬇️
...in/scala/com/salesforce/op/cli/CommandParser.scala	`0% <0%> (-98.12%)`	⬇️
...cala/com/salesforce/op/cli/gen/ProblemSchema.scala	`0% <0%> (-96.56%)`	⬇️
...src/main/scala/com/salesforce/op/cli/gen/Ops.scala	`0% <0%> (-94%)`	⬇️
...ain/scala/com/salesforce/op/cli/SchemaSource.scala	`0% <0%> (-87.94%)`	⬇️
...a/com/salesforce/op/cli/gen/ProjectGenerator.scala	`0% <0%> (-87.5%)`	⬇️
...in/scala/com/salesforce/op/cli/CliParameters.scala	`0% <0%> (-80.96%)`	⬇️
...src/main/scala/com/salesforce/op/cli/CliExec.scala	`0% <0%> (-80%)`	⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05963bc...2f7b5e0. Read the comment docs.

tovbinm · 2019-02-25T23:35:22Z

@rajdeepd please move the notebook files into helloworld/notebooks folder

rajdeepd · 2019-02-26T17:04:47Z

@tovbinm done

tovbinm · 2019-02-26T18:12:46Z

Please clean the notebooks from the evaluation results and just keep the commands.
In between the commands please copy over the docs from here - https://docs.transmogrif.ai/en/stable/examples/index.html for each notebook.

rajdeepd · 2019-03-01T13:39:31Z

@tovbinm made the changes

tovbinm · 2019-03-01T21:57:05Z

helloworld/notebooks/ListOfFiles.ipynb

@@ -0,0 +1,128 @@
+{


why do we need ListOfFiles.ipynb?

test notebook to make sure training data is mounted in the container path, will remove it

rajdeepd · 2019-03-05T17:59:10Z

@tovbinm please review

leahmcguire

This is awesome!! Thanks for contributing!

leahmcguire · 2019-03-07T00:21:22Z

helloworld/notebooks/README.md

+
+```bash
+docker run -p 8888:8888 -v /Users/rdua/work/github/rajdeepd/TransmogrifAI/helloworld/notebooks:/home/beakerx/helloworld-notebooks \
+-v /Users/rdua/work/github/rajdeepd/TransmogrifAI/helloworld:/home/beakerx/helloworld beakerx/beakerx


maybe just change example to $TransmografaiPATH

leahmcguire · 2019-03-07T00:23:32Z

helloworld/notebooks/OpTitanicSimple.ipynb

+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After model has been fitted we use `scoreAndEvaluate()` function to evaluate the metrics"


it would be nice to mention that you can change out the data before doing this by either setting a new path or a new reader

leahmcguire · 2019-03-07T00:24:18Z

helloworld/notebooks/OpIris.ipynb

+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After model has been fitted we use scoreAndEvaluate() function to evaluate the metrics"


it would be nice to mention that you can change out the data before doing this by either setting a new path or a new reader

tovbinm · 2019-03-07T23:33:36Z

helloworld/notebooks/README.md

@@ -0,0 +1,100 @@
+# Transmogrify on Jupyter


TransmogrifAI on Jupyter

tovbinm · 2019-03-07T23:33:44Z

helloworld/notebooks/README.md

@@ -0,0 +1,100 @@
+# Transmogrify on Jupyter
+
+In this section we will look at how Transmogrify can be run within Scala notebooks on


TransmogrifAI

tovbinm · 2019-03-07T23:34:06Z

helloworld/notebooks/README.md

+In this section we will look at how Transmogrify can be run within Scala notebooks on
+Jupyter.
+
+We are going to leverage [BeakerX](http://beakerx.com/) scala kernel for Jupyter


Scala kernel

tovbinm · 2019-03-07T23:35:07Z

helloworld/notebooks/README.md

+
+* Apache Maven
+* Python 3
+* JDK 8 (JDK 10 or above can cause issues with Transmogrify)


simply JDK 8 only - don't mention anything else

tovbinm · 2019-03-07T23:35:25Z

helloworld/notebooks/README.md

+Installation using pip
+
+```$xslt
+sudo pip install beakerx


does this have to be sudo? (just curios)

sometimes pip install fails because of permissions issue

tovbinm · 2019-03-07T23:37:53Z

helloworld/notebooks/README.md

+BeakerX provides a [docker container image](https://hub.docker.com/r/beakerx/beakerx/) on docker hub. 
+
+Assuming your Transmogrify source code is downloaded at `/Users/rdua/work/github/rajdeepd/TransmogrifAI`. You can use 
+the following command to start the container. We need the directory above so that we can mount sample notebooks and dataset


Which command are you referring to?

docker run command - updated the doc

tovbinm · 2019-03-07T23:38:33Z

helloworld/notebooks/README.md

+
+### Set TransmogrifaiPATH
+
+export TransmogrifaiPATH=<transmogrify installation dir>


escape with `

export TransmogrifaiPATH=<TransmogrifAI installation dir>

tovbinm · 2019-03-07T23:39:35Z

helloworld/notebooks/README.md

+### Run the beakerx Container 
+
+```bash
+docker run -p 8888:8888 -v $TransmogrifaiPATH/helloworld/notebooks:/home/beakerx/helloworld-notebooks \


why do we need to mount both paths?

one is for jupyter notebooks, another for data -- data is helloworld so the jupyter notebook mount point cannot access it

tovbinm · 2019-03-07T23:39:58Z

helloworld/notebooks/README.md

+
+On opening the image in the browser you will notice that in the home page
+
+![notebook_home][notebooks_home]


missing image notebook_home?

tovbinm · 2019-03-07T23:41:26Z

helloworld/notebooks/README.md

+
+#### OpTitanicSimple
+
+[OpTitanicSimple.ipynb](http://localhost:8888/notebooks/helloworld-jupyter/OpTitanicSimple.ipynb)


are you sure that the links for the notebooks are correct?

thanks for pointing out..

tovbinm · 2019-03-07T23:59:29Z

helloworld/notebooks/OpTitanicSimple.ipynb

+   "outputs": [],
+   "source": [
+    "val fittedWorkflow = workflow.train()\n",
+    "println(s\"Summary: ${fittedWorkflow.summary()}\")"


println(s\"Summary: ${fittedWorkflow.summaryPretty()

tovbinm · 2019-03-07T23:59:39Z

helloworld/notebooks/OpIris.ipynb

+   "outputs": [],
+   "source": [
+    "val fittedWorkflow = workflow.train()\n",
+    "println(s\"Summary: ${fittedWorkflow.summary()}\")"


same here - println(s\"Summary: ${fittedWorkflow.summaryPretty()

leahmcguire

LGTM!

rajdeepd · 2019-03-09T03:16:27Z

@tovbinm @leahmcguire please merge

tovbinm · 2019-03-11T07:02:19Z

Thank you @rajdeepd

salesforce-cla · 2020-12-11T21:39:00Z

Thanks for the contribution! Unfortunately we can't verify the commit author(s): Leah McGuire <l***@s***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, refresh the status of this Pull Request.

rajdeepd requested review from leahmcguire and tovbinm as code owners February 24, 2019 15:16

rajdeepd force-pushed the jupyter branch 3 times, most recently from 53e0571 to 05215ef Compare February 26, 2019 10:26

rajdeepd force-pushed the jupyter branch from 044e4be to 7161a87 Compare March 1, 2019 12:14

tovbinm reviewed Mar 1, 2019

View reviewed changes

rajdeepd force-pushed the jupyter branch from 6966f55 to df3c9af Compare March 2, 2019 01:05

leahmcguire reviewed Mar 7, 2019

View reviewed changes

rajdeepd force-pushed the jupyter branch from 305618a to c84758f Compare March 7, 2019 14:59

tovbinm reviewed Mar 7, 2019

View reviewed changes

jupyter notebooks for transmogrify samples

916e28d

rajdeepd force-pushed the jupyter branch from 41bd221 to 916e28d Compare March 8, 2019 18:02

leahmcguire approved these changes Mar 8, 2019

View reviewed changes

Merge branch 'master' into jupyter

f786cb9

tovbinm merged commit 6a593f6 into salesforce:master Mar 11, 2019

tovbinm mentioned this pull request Apr 10, 2019

Release 0.5.2 #277

Merged

tovbinm mentioned this pull request Jul 11, 2019

Release 3.3.3 #26

Merged

salesforce-cla bot added the cla:missing label Dec 11, 2020

		@@ -0,0 +1,100 @@
		# Transmogrify on Jupyter

		In this section we will look at how Transmogrify can be run within Scala notebooks on


		### Set TransmogrifaiPATH

		export TransmogrifaiPATH=<transmogrify installation dir>


		On opening the image in the browser you will notice that in the home page

		![notebook_home][notebooks_home]


		#### OpTitanicSimple

		[OpTitanicSimple.ipynb](http://localhost:8888/notebooks/helloworld-jupyter/OpTitanicSimple.ipynb)

Conversation

rajdeepd commented Feb 24, 2019

Uh oh!

codecov bot commented Feb 24, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codecov bot commented Feb 24, 2019

Codecov Report

Uh oh!

tovbinm commented Feb 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rajdeepd commented Feb 26, 2019

Uh oh!

tovbinm commented Feb 26, 2019

Uh oh!

rajdeepd commented Mar 1, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rajdeepd commented Mar 5, 2019

Uh oh!

leahmcguire left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Feb 24, 2019 •

edited

Loading

tovbinm commented Feb 25, 2019 •

edited

Loading