Julia language is not ready (2018)

Update: I have picked up Julia again in 2020 and I am enjoying it.

Beyond the core math and functions, which work fine, trying to run any of the 2200 packages in Julia 1.0.1 is a night full of frustrations.

I have tried countless code examples and most of them fail.

The most important to me "using PyCall" was the final signal for me that Julia and I need a little time apart. Maybe when some percentage of packages make it from 0.1 and 0.3 to something like 0.9 and 1.0 then Julia will be more usable.

import Pkg;

Pkg.add("PyCall")

Updating registry at `~/.julia/registries/General` Updating git-repo `https://github.com/JuliaRegistries/General.git` [1mFetching: [========================================>] 100.0 %.0 %

Resolving package versions... 

Updating `~/.julia/environments/v1.0/Project.toml` [5c1ef01f] 

- Cookbook v0.3.0 [5befdd30] - Hilbert v0.1.0 [5c077215] 

- LatticeSites v0.2.0 Updating `~/.julia/environments/v1.0/Manifest.toml` [5c1ef01f] 

- Cookbook v0.3.0 [5befdd30] 

- Hilbert v0.1.0 [5c077215] 

- LatticeSites v0.2.0

Pkg.update()

Updating registry at `~/.julia/registries/General` 

Updating git-repo `https://github.com/JuliaRegistries/General.git` 

Resolving package versions... 

Installed KernelDensity ── v0.5.1 

Installed Interpolations ─ v0.9.2 

Updating `~/.julia/environments/v1.0/Project.toml` [5cadff95] 

- JuliennedArrays v0.1.0 [5ab0869b] ↑ KernelDensity v0.5.0 ⇒ v0.5.1 [5c958174] 

- SuperEnum v0.1.3 Updating `~/.julia/environments/v1.0/Manifest.toml` [a98d9a8b] ↓ Interpolations v0.10.5 ⇒ v0.9.2 [5cadff95] 

- JuliennedArrays v0.1.0 [5ab0869b] ↑ 

KernelDensity v0.5.0 ⇒ v0.5.1 [5c958174] 

- SuperEnum v0.1.3

using PyCall

SYSTEM: show(lasterr) caused an error Stacktrace: [1] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:630 [2] macro expansion at ./logging.jl:312 [inlined] [3] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:701 [4] _require(::Base.PkgId) at ./loading.jl:934 [5] require(::Base.PkgId) at ./loading.jl:855 [6] macro expansion at ./logging.jl:311 [inlined] [7] require(::Module, ::Symbol) at ./loading.jl:837 [8] top-level scope at In[1]:1

KERNEL EXCEPTION
SYSTEM: show(lasterr) caused an errorStacktrace:
[1] #invokelatest#1 at ./essentials.jl:697 [inlined]
[2] invokelatest at ./essentials.jl:696 [inlined]
[3] eventloop(::ZMQ.Socket) at /Users/uki/.julia/packages/IJulia/0cLgR/src/eventloop.jl:8
[4] (::getfield(IJulia, Symbol("##12#15")))() at ./task.jl:259

The error above prevents the execution of the below:

@pyimport math 

#math.sin(math.pi / 4) - sin(pi / 4) # returns 0.0

@pyimport time

@pyimport numpy.random as nr nr.rand(3,4)

Jupyter Error 49

$ jupyter notebook

Traceback (most recent call last):

File "/anaconda3/bin/jupyter-notebook", line 11, in


sys.exit(main())

File "/anaconda3/lib/python3.7/site-packages/jupyter_core/application.py", line 266, in launch_instance

return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)

File "/anaconda3/lib/python3.7/site-packages/traitlets/config/application.py", line 657, in launch_instance

app.initialize(argv)

File "", line 2, in initialize

File "/anaconda3/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error

return method(app, *args, **kwargs)

File "/anaconda3/lib/python3.7/site-packages/notebook/notebookapp.py", line 1602, in initialize

self.init_webapp()

File "/anaconda3/lib/python3.7/site-packages/notebook/notebookapp.py", line 1381, in init_webapp

self.http_server.listen(port, self.ip)

File "/anaconda3/lib/python3.7/site-packages/tornado/tcpserver.py", line 143, in listen

sockets = bind_sockets(port, address=address)

File "/anaconda3/lib/python3.7/site-packages/tornado/netutil.py", line 168, in bind_sockets

sock.bind(sockaddr)

OSError: [Errno 49] Can't assign requested address

Short Answer

$ jupyter notebook --ip=127.0.0.1

PORT 8888

You can also explore what you are already running on PORT 8888

$ lsof | grep 8888Skype      407  uki  txt       REG                1,5   

$ lsof -nP | grep 8888

Julia

Download Julia

https://julialang.org/downloads/

For me, MacOS Package (.dmg)

               _

   _       _ _(_)_     |  Documentation: https://docs.julialang.org  (_)     | (_) (_)    |   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.  | | | | | | |/ _` |  |  | | |_| | | | (_| |  |  Version 1.0.1 (2018-09-29) _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release|__/                   |
julia> println("Hello World,", "Julia!")Hello World,Julia!

Clone Julia:

git clone https://github.com/JuliaLang/julia.git

Visit JuliaBox.com (iJulia Jupyter notebooks):

JuliaBox.com

Install Juno (Atom IDE)

http://docs.junolab.org/latest/man/installation.html

iJulia in Jupyter Notebook

julia> using Pkg
julia> Pkg.add("IJulia")

Jupyter NBInclude.jl

https://github.com/stevengj/NBInclude.jl

$ git clone https://github.com/stevengj/NBInclude.jl.git julia_NBInclude

Windows 10 UBuntu bash

My company uses mostly Windows and Ubuntu servers, I requested a MacBook, obviously.

However, there are certain internally-written programs that will run on the Windows only, so I run Parallels hypervisor with Windows 10 and Ubuntu.

On Windows 10, I have a hard time with the Command Prompt, last time I used DOS was in c. 1999, so I opted to try installing Bash.

$ ping Google.com

PING Google.com (74.125.138.102) 56(84) bytes of data.
64 bytes from yi-in-f102.1e100.net (74.125.138.102): icmp_seq=1 ttl=128 time=30.4 ms

$ ssh dummy@74.125.138.102

ssh: connect to host 74.125.138.102 port 22: Connection refused

Checking for Python

$ python --version

Command 'python' not found, but can be installed with:

sudo apt install python3
sudo apt install python
sudo apt install python-minimal

You also have python3 installed, you can run 'python3' instead.

$ python3 --version

Python 3.6.5

$ python3

Python 3.6.5 (default, Apr  1 2018, 05:46:30)

[GCC 7.3.0] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> print("Hello World")

Hello World

>>> exit()

$

Attempting to get Anaconda

wget https://3230d63b5fc54e62148e-c95ac804525aac4b6dba79b00b39d1d3.ssl.cf1.rackcdn.com/Anaconda-2.3.0-Linux-x86_64.sh
$ chmod +x Anaconda-2.3.0-Linux-x86_64.sh
$ . Anaconda-2.3.0-Linux-x86_64.sh

-bash: .: Anaconda-2.3.0-Linux-x86_64.sh: cannot execute binary file

$ java

Command 'java' not found, but can be installed with:

$ sudo apt install default-jre
$ sudo apt install openjdk-11-jre-headless
$ sudo apt install openjdk-8-jre-headless
$ sudo apt install openjdk-11-jre-headless

$ java --version

openjdk 10.0.1 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Ubuntu-3ubuntu1)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Ubuntu-3ubuntu1, mixed mode)

Amazon remembers 30 of my past mobile devices!

It is funny, Amazon remembers 30 of my past mobile devices!

I had no idea until I have clicked for a new book to be delivered to my Kindle app.

Can you say I have been into mobile development since 2009?

graphlab.canvas.set_target('ipynb') error

I am getting an error in the following line:

graphlab.canvas.set_target('ipynb') # alternative 'browser', port=8889

/Volumes/DATA/anaconda3/envs/py27/lib/python2.7/site-packages/graphlab/canvas/server.pyc

108 self.__server = tornado.httpserver.HTTPServer(self.__application, io_loop=self.__loop)

TypeError: initialize() got an unexpected keyword argument 'io_loop'

Fix Attempt 1:
 

$ conda update tornado

environment location: /Volumes/DATA/anaconda3/envs/py27

tornado: 5.1-py27h1de35cc_0 --> 5.1.1-py27h1de35cc_0
Fix is not successful



Fix Attempt 2:
Reinstall Anaconda3

Install Anaconda Python environment with Jupyter Notebook

This post was updated on September 29, 2022

Check if you have Conda installed

% conda update --all -y
>> command not found: conda

Install Anaconda Python

I prefer an installer on my laptop: 

https://www.anaconda.com/

After installation, make sure you restart the Terminal (control N).

Note, Anaconda includes the following and much more:

• curl

• numpy

• matplotlib

• jupyter_core
  

• protobuf

• sqlite

• ipython

• jupyterlab

• jupyter

• notebook

• matplotlib

• pip

• pandas

• pillow

• scikit-learn

• python-3.9

Update Conda

$ conda activate base

$ conda update -n base -c defaults conda

$ conda update --all -y

How to start Jupyter Notebook

cd $REPO // the directory you want as a base of your project (e.g. in GitHub directory)  
jupyter-lab

This will start in the browser: http://localhost:8888/lab

That is all you need to start working.
Alternatively, you may want to create ENVIRONMENT specific installation. This is useful if you work on multiple projects, especially over a long time when LIBRARIES change and code becomes outdated.

Check Python Version that came with Conda

% python --version
Python 3.9.13

 

Create a Conda TensorFlow environment

$ conda create -n py_39_tf python=3.9 tensorflow -y

This installs tensorflow         pkgs/main/osx-64::tensorflow-2.9.1

List conda environments you already created

$ conda info --envs

 

% conda info --envs

# conda environments:

#

base                  *  /Users/uki/opt/anaconda3

py_39_tf                 /Users/uki/opt/anaconda3/envs/py_39_tf

Activate the Conda Environment you want Jupyter in.

% conda activate py_39_tf

(py_39_tf) uki ~ %

Install new Jupyther kernel

$ python -m ipykernel install --user --name py_36_tf --display-name "Python 3.6 (tensorflow)"

 

List currently installed Jupyther kernels 

$ ls -alt ~/Library/Jupyter/kernels/

total 0
drwx------ 7 uki staff 224 Nov 22 11:50 ..
drwxr-xr-x 9 uki staff 288 Nov 22 11:43 .
drwxr-xr-x 5 uki staff 160 Nov 4 06:35 julia-1.4
drwxr-xr-x 5 uki staff 160 Apr 1 2020 python361064bitpy36condaa60168e76a7b4349b469299762ee4c30
drwxr-xr-x 5 uki staff 160 Apr 1 2020 python38264bitpytorchcondaaf5a833263b448b8b2738bb5a7355c8a
drwxr-xr-x 5 uki staff 160 Apr 1 2020 python361064bitturiconda565ecc262d0845fbb235ae21ac24296f

Installed kernelspec py_36_tf in /Users/uki/Library/Jupyter/kernels/py_36_tf

Refresh a page with Jupyther Lab notebook and change to the new kernel.

Delete kernels that you want to replace

$ rm -r ~/Library/Jupyter/kernels/my_old_kernel_name

 

Microsoft Excel spreadsheet creates 1,048,576 rows!

I am sorry Microsoft, I really thought you have improved, but sometimes you just amaze me...

I bought my own Office license, so I am a paying customer.

The Excel creates 1,048,576 rows and yes, this brings the spreadsheet to a holt.

I am not making this up, there is an official post on it:

https://support.office.com/en-us/article/Locate-and-reset-the-last-cell-on-a-worksheet-C9E468A8-0FC3-4F69-8038-B3C1D86E99E9

The question is why in the world would you do it to a new spreadsheet with a few rows of actual data?

Force-killing frozen Excel

Re-opening the Excel reveals the same condition.

Select and Delete All Empty Rows

You can select and delete all rows below your last row with data.
- Select the first EMPTY row.
Depending on your keyboard you can use the following key combination:
- SHIFT + CONTROL + Arrow DOWN
or
- Control + END
- Right click and select DELETE

IT WORKS, then it recreated 1,048,576 rows!
The developer who created this functionality should be forced to do some very unpleasant things for each row created.

Hiding millions of rows and columns

Some people suggest to select and hide the offending rows and columns (to the right).
I have used it, but I think it is stupid, the rows are still there.

Select and Clear All Empty Rows

Since the cells were auto-generated, I am not sure why the article is suggesting this, but I tried this too.

- Select all empty cells (same as above)

- Right click and "Clear Contents"

- Save file

- Close

- Open file

No difference

Try #2

- Select all empty cells (same as above)

- Home > Clear > Clear All

- Save file

- Close

- Open file

No difference

The file on the disk is 29.5 MB

The Best Solution

- Select ALL POPULATED cells

- Open Apple Numbers

- paste cells

- Export file to Excel when you have to send it to coworkers

The file on the disk is 8 kB

Please share and donate!

Jupyter TOC

In order to create a Table of Contents in Jupyter Notebook, you need anchor tags above the headings:

replace_record
">

## Table of Contents

1. [Reading the CSV file](#Reading-CSV-file)
1. [The article's HTML template](#template)
1. [Main part - replace for every record](#replace_record)

Chrome Refresh

Caching of old content in the browser can drive you nuts when you develop HTML pages.

On Mac Google Chrome you refresh bypassing the cache by:

Holding both the ⌘ Cmd and ⇧ Shift keys and pressing the R key.

PCA9685

pip install Adafruit-PCA9685

References:

https://learn.adafruit.com/adafruit-16-channel-servo-driver-with-raspberry-pi/overview

L298N H-bridge Motor Controller

Connecting to Raspberry Pi Zero

Ubiquity Robotics Ubuntu Pi image for Pi with ROS

Ubiquity Robotics Lubuntu Pi image for Pi


On Ubuntu OS:
sudo apt install gnome-disk-utility

On Mac OS:
https://etcher.io/

Raspberry Pi 3 Model B
User :Ubuntu
Password: ubuntu (please change)




Pi HotSpot:

SSID is ubiquityrobotXXXX
where XXXX is part of the MAC address
Password: robotseverywhere

$ ssh ubuntu@192.168.50.16

The authenticity of host '192.168.50.16 (192.168.50.16)' can't be established.

ECDSA key fingerprint is SHA256:xq7SSuVwuVdifKaZGNqgt09LiDiZ0cnayd1tp4kWzD4.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.50.16' (ECDSA) to the list of known hosts.

ubuntu@192.168.50.16's password:

Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.9.80-v7+ armv7l)

* Documentation: https://help.ubuntu.com

* Management: https://landscape.canonical.com

* Support: https://ubuntu.com/advantage

464 packages can be updated.

140 updates are security updates.
...

How to update Ubuntu from the command line?

ubuntu@ubiquityrobot:~$ sudo apt-get update
[sudo] password for ubuntu:


ubuntu@ubiquityrobot:~$ sudo apt-get upgrade

E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?

Disable the Ubiquity Robotics scripts (unless you run their robots):

$ sudo systemctl disable magni-base
Removed symlink /etc/systemd/system/multi-user.target.wants/magni-base.service.

References

https://ubuntu-pi-flavour-maker.org/

Ubiquity Robotics Lubuntu Pi image for Pi:

https://downloads.ubiquityrobotics.com/pi.html

Python template

To replace tokens in the HTML template I use method str.format()

line3 = '      {ARTICLE_TITLE}'
line3 = line3.format(ARTICLE_URL="http...", ARTICLE_TITLE="My Note")
print( line3 )

My Note'

GitHub gitignore

Sometimes the .gitignore file does not work with GitHub, to fix it I execute the following:



$ git rm -r --cached . 

$ git commit -a -m "fixed .gitignore"

Remove commit

Sometimes when you COMMIT to GitHub a file larger than 100MB, the PUSH is rejected.


Your branch is ahead of 'origin/master' by 5 commits.
You have to remove the last (or more) COMMIT, the following command does not change your code:

git reset --soft HEAD~1

Then fix the problem, usually by marking the large files in .gitignore

$ git commit -a -m "added gitignore with _DATA/* directory"


$ git status

Untracked files:
(use "git add ..." to include in what will be committed)

src/_DATA/amazon_baby.csv
src/_DATA/amazon_baby.gl/
src/_DATA/home_data.gl/
src/_DATA/image_test_data/
src/_DATA/image_train_data/
src/_DATA/people-example.csv
src/_DATA/people_wiki.csv
src/_DATA/people_wiki.gl/
src/_DATA/song_data.csv
src/_DATA/song_data.gl/


Then push again and pray:

$ git push

 $ git status

On branch master

Your branch is up to date with 'origin/master'

GitHub SSH

Go to:

https://github.com/settings/keys

Read:

https://gist.github.com/developius/c81f021eb5c5916013dc


$ ssh -T git@github.com


The authenticity of host 'github.com (192.30.253.112)' can't be established.

RSA key fingerprint is SHA256:nTh....

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'github.com,1xx.xx.xx.xx' (RSA) to the list of known hosts.

Enter passphrase for key '/Users/uki/.ssh/id_rsa':

Hi UkiDLucas! You've successfully authenticated, but GitHub does not provide shell access.

Find files that are too big for GitHub

Command explanation

• Find
• in current directory
• files bigger than 50 MB, (100Mb is too big for GitHub)
• edited within 365 days
• the type will be a file
• execute list files
• list format
• human readable
• sort by Size (not working)

$ find . -size +50M -mtime +365 -type f -exec ls -lhS {} \;

-rw-r--r--@ 1 uki  admin    49M Nov  6  2016 ./Coursera/UW/ML/Week3/amazon_baby.gl/m_bfaa91c17752f745.0000

-rw-r--r--@ 1 uki  admin    69M Nov  6  2016 ./Coursera/UW/ML/Week4/people_wiki.gl/m_4549381c276b46c6.0000

-rw-r--r--@ 1 uki  admin    60M Nov  6  2016 ./Coursera/UW/ML/Week5/song_data.gl/m_cccc16853452d1ed.0000

-rw-r--r--@ 1 uki  admin    63M Nov  6  2016 ./Coursera/UW/ML/Week6/image_test_data/m_e16f5ffd2c088370.0000

-rw-r--r--@ 1 uki  admin    31M Nov  6  2016 ./Coursera/UW/ML/Week6/image_train_data/m_504edbda459b24ff.0000

ROS Mac

http://wiki.ros.org/kinetic/Installation/OSX/Homebrew/Source

ROS Droid

Creating Android ROS nodes to:
- add control UI (HMI)
- utilize existing phone sensors:
-- gyroscope
-- GPS
-- compass
-- camera
- do some data processing (flagship phones are very powerful)
- use TensorFlow AI

References:
https://github.com/rosjava/android_core

http://wiki.ros.org/rosjava/Tutorials/Create%20a%20ROS%20Android%20Node

HiKey 960 ver.B Single Board Computer (SBC) 4GB

Specifications:

• Kirin 960 SOC, Big.Little architecture
•     4 Cortex A73 CPU
•     4 Cortex A53 CPU 
• GPU: Mali G71 MP8 (high image processing)
• 4GB LPDDR4 DRAM
• 32GB UFS flash storage
• Power: 12V@2A power supply (8v to 18V)
• DC jack (EIAJ-3 compliant)
• 4.75mm outer diameter
• 1.7mm center pin (positive)
• 2 x USB 3.0 Type-A
• 1 x USB Type-C OTG (if anything is plugged-in the other USB stop working)
• Audio/Video OUT: HDMI 1.4 ( Type A-full) monitor connector
• Micro SD card
• Wi-Fi TIWL1837MOD 2.4GHz, 5GHz (dual-band, dual antenna)
• Bluetooth
• PCIe Gen 2 M.2 M key connector
• External boot switch
• Boot mode switch
• Auto power switch
• Power reset button
• 40-pin connector (low speed): 
• I2S, 
• 2 x UART, 
• 2 x I2C
• SPI
• 12 x GPIO
• DC power
• 60-pin HS (high-speed) connector
• 4L-MIPI DSI
• I2C x2
• SPI (48M)
• USB 2.0
• 2L+4L MIPI CSI
• OS: Android / Linux

Getting Started page:
https://www.96boards.org/documentation/consumer/hikey/hikey960/getting-started/


Factory Image (latest hikey960 linaro):
~ 600MB zip
local storage: /Volumes/DATA/linaro/HiKey/image/hikey960-linaro-2018.10.04-factory-ccfcf697.zip
https://snapshots.linaro.org/96boards//hikey960/linaro/aosp-master/latest/

Good video on building AOSP for HiKey 960:

https://youtu.be/RD_NZRKiZBU

Android Code Tree:
https://gitlab.com/HiKey960-Car/android_device_linaro_hikey/tree/master_091118_car96


When USB C is connected, neither of USB A will work:
https://bugs.96boards.org/show_bug.cgi?id=500#c1

DBPedia

Exploring..


https://wiki.dbpedia.org/

Jupyter notebook TOC

https://github.com/shoval/jupyter-navbar

Python Dictionary (KEY, VALUE pairs) operations

# define empty dictionary
special_words_count = {}


# populate dictionary with sums
special_words_count["awesome"] = products["awesome"].sum()
special_words_count["great"] = products["great"].sum()
special_words_count["fantastic"] = products["fantastic"].sum()
special_words_count["amazing"] = products["amazing"].sum()
special_words_count["love"] = products["love"].sum()
special_words_count["horrible"] = products["horrible"].sum()
special_words_count["bad"] = products["bad"].sum()
special_words_count["terrible"] = products["terrible"].sum()
special_words_count["awful"] = products["awful"].sum()
special_words_count["wow"] = products["wow"].sum()
special_words_count["hate"] = products["hate"].sum()



# print values in descending order
# key=lambda x: x[1] # gets the VALUE (i.e. the second [0,1] element) of the PAIR x.
# reverse=True - set descending order
for key, value in sorted(special_words_count.items(), key=lambda x: x[1], reverse=True):
    print value, key

42420 great
40277 love
3197 bad
2002 awesome
1305 amazing
1057 hate
873 fantastic
673 terrible
659 horrible
345 awful
131 wow

Another example of sorting the Dictionary

print name_value_dictionary

{'fantastic': 0.8913030903042991, 'love': 1.3998983430174596, 'bad': -0.9858273699287683, 'awesome': 1.0580088887848043, 'great': 0.8839378948979554, 'terrible': -2.090499984872614, 'amazing': 0.8928024225084649, '(intercept)': 1.367283152293053, 'horrible': -1.9965180055878113, 'awful': -1.7646995563132248, 'hate': -1.4091640627563906, 'wow': -0.0541450123332599}



def input_method(my_tuple):
    #print "*", my_tuple[1] # sanity check
    return my_tuple[1]

tuples = name_value_dictionary.items() # key: value

for key, value in sorted(tuples, key=input_method, reverse=True):
    print value, key



1.39989834302 love
1.36728315229 (intercept)
1.05800888878 awesome
0.892802422508 amazing
0.891303090304 fantastic
0.883937894898 great
-0.0541450123333 wow
-0.985827369929 bad
-1.40916406276 hate
-1.76469955631 awful
-1.99651800559 horrible
-2.09049998487 terrible