Julia language: DataFrames: insert a column

 The following code 

  1. inserts a new column into DataFrame named df
  2. inserts this column as a first (col_ind =1) column
  3. names this column Rata_Die
  4. populates the column values with an array of Int64 zeros (0)
  5. makes sure the name of the column is unique



col_ind = 1

insertcols!(df, col_ind, :Rata_Die => zeros(Int64, record_count); makeunique = true )



As an Amazon Associate I earn from qualifying purchases.

Julia language UNIX date time

Unix time date format is used in many applications, including Yahoo finance.



using Dates, Printf

unix_date = @sprintf("%.0f", Dates.datetime2unix(Dates.now()))

println( unix_date ) # String



Output:

1603013077


As an Amazon Associate I earn from qualifying purchases.

Opening the Jupyter notebook in a different drive on Windows 10.

Assuming that you have Anaconda 3 installed, 
you can open the "Anaconda Prompt" window and start jupyter,
however, it opens in the default C: drive.

Anaconda prompt does not allow you to change the drive from the command line. (WTF !?!)

The simplest solution I found was to specify the new drive (R: for Repos) in Jupyter startup:

> jupyter notebook --notebook-dir 'R:'

Sidenote: 

In case you wonder why I am putting up with Windows 10 and not working on my MacOS, I wanted to try the NVidia GPU performance with Julia in the Jupyter notebook. 



As an Amazon Associate I earn from qualifying purchases.

Julia DataFrames

 

Filtering DataFrame


New Syntax

 

Syntax explained:

  1. from original "data" (type DataFrames)
  2. create new "dataA" (type DataFrames)
  3. filter indices in column "Treatment" with values equal to "A"
  4. include ALL COLUMNS indicated by ":"
  5. Show the first 6 rows


dataA = data[isequal.(data.Treatment, "A"), : ] 

first(dataA, 6)



6 rows × 5 columns

AgeWCCCRPTreatmentResult
Int64Float64Int64StringString
13710.230AWorse
26710.470AWorse
34210.4100AStatic
45612.580AWorse
5609.50AStatic
66213.710AWorse

 


Deprecated Syntax

# dataA = data[data[:Treatment] .== "A", :] 

 

Warning: `getindex(df::DataFrame, col_ind::ColumnIndex)` is deprecated, use `df[!, col_ind]` instead.





As an Amazon Associate I earn from qualifying purchases.

Julia packages (Pkg)

 Here are the Julia packages I have been using so far, I keep this list in case I need to re-install Julia.

Un-comment, install and pre-compile the package you need:


import Pkg

# Pkg.add("Atom") 

# using Atom

 

# Pkg.add("BinDeps") 

# using BinDeps

 

# Pkg.add("CSV") 

# using CSV

 

# Pkg.add("CUDAapi") 

# using CUDAapi

 

# Pkg.add("Calculus") 

# using Calculus

 

# Pkg.add("CodecZlib") 

# using CodecZlib

 

# Pkg.add("Codecs") 

# using Codecs

 

# Pkg.add("ColorTypes") 

# using ColorTypes

 

# Pkg.add("Colors") 

# using Colors

 

# Pkg.add("Combinatorics") 

# using Combinatorics

 

# Pkg.add("Compat") 

# using Compat

 

# Pkg.add("Compose") 

# using Compose

 

# Pkg.add("Contour") 

# using Contour

 

# Pkg.add("DataArrays") 

# using DataArrays

 

# Pkg.add("DataFrames") 

# using DataFrames

 

# Pkg.add("DataStructures") 

# using DataStructures

 

# Pkg.add("Distances") 

# using Distances

 

# Pkg.add("Distributions") 

# using Distributions

 

# Pkg.add("FileIO") 

# using FileIO

 

# Pkg.add("FixedPointNumbers") 

# using FixedPointNumbers

 

# Pkg.add("Flux") 

# using Flux

 

# Pkg.add("ForwardDiff") 

# using ForwardDiff

 

# Pkg.add("GLM") 

# using GLM

 

# Pkg.add("GR") 

# using GR

 

# Pkg.add("GZip") 

# using GZip

 

# Pkg.add("GeometryBasics") 

# using GeometryBasics

 

# Pkg.add("Hexagons") 

# using Hexagons

 

# Pkg.add("HypothesisTests") 

# using HypothesisTests

 

# Pkg.add("IJulia") 

# using IJulia

 

# Pkg.add("JSON") 

# using JSON

 

# Pkg.add("Juno") 

# using Juno

 

# Pkg.add("KernelDensity") 

# using KernelDensity

 

# Pkg.add("Knet") 

# using Knet

 

# Pkg.add("Lathe") 

# using Lathe

 

# Pkg.add("Loess") 

# using Loess

 

# Pkg.add("Measures") 

# using Measures

 

# Pkg.add("Metalhead") 

# using Metalhead

 

# Pkg.add("NaNMath") 

# using NaNMath

 

# Pkg.add("OffsetArrays") 

# using OffsetArrays

 

# Pkg.add("Optim") 

# using Optim

 

# Pkg.add("PDMats") 

# using PDMats

 

# Pkg.add("Parameters") 

# using Parameters

 

# Pkg.add("Plots") 

# using Plots

 

# Pkg.add("Polynomials") 

# using Polynomials

 

# Pkg.add("Primes") 

# using Primes

 

# Pkg.add("PyCall") 

# using PyCall

 

# Pkg.add("PyPlot") 

# using PyPlot

 

# Pkg.add("QuartzImageIO") 

# using QuartzImageIO

 

# Pkg.add("RDatasets") 

# using RDatasets

 

# Pkg.add("Reexport") 

# using Reexport

 

# Pkg.add("Rmath") 

# using Rmath

 

# Pkg.add("Roots") 

# using Roots

 

# Pkg.add("Showoff") 

# using Showoff

 

# Pkg.add("SortingAlgorithms") 

# using SortingAlgorithms

 

# Pkg.add("StaticArrays") 

# using StaticArrays

 

# Pkg.add("StatsBase") 

# using StatsBase

 

# Pkg.add("StatsFuns") 

# using StatsFuns

 

# Pkg.add("StatsPlots") 

# using StatsPlots

 

# Pkg.add("TensorFlow") 

# using TensorFlow

 

# Pkg.add("TextAnalysis") 

# using TextAnalysis

 

# Pkg.add("TinySegmenter") 

# using TinySegmenter

 

# Pkg.add("URIParser") 

# using URIParser

 

# Pkg.add("UnicodePlots") 

# using UnicodePlots

 

# Pkg.add("WordTokenizers") 

# using WordTokenizers

 

# Pkg.add("Dates") 

# using Dates

 

# Pkg.add("DelimitedFiles") 

# using DelimitedFiles

 

# Pkg.add("SHA") 

# using SHA

 

# Pkg.add("Statistics") 

# using Statistics



As an Amazon Associate I earn from qualifying purchases.

"Rata Die". Counting Days since...

 

Counting Days since...

https://en.wikipedia.org/wiki/Rata_Die



# col1 is an exported column of dates

lastDay = 54 # last row in the column sorted by the date

daysSince(x) = Dates.datetime2rata(x) - Dates.datetime2rata(col1[lastDay])

arrayDays = Array{Int64}(undef, lastDay) # array size is 54

for i = 1:lastDay

    arrayDays[i] = daysSince(col1[i])

end

arrayDays


Output:

54-element Array{Int64,1}:
613
606
599
592
582
575
568
554
547
540
533
526
512


As an Amazon Associate I earn from qualifying purchases.

Flux: The Julia Machine Learning Library

 

Flux is a machine learning library writer in Julia language.



Installation

(v1.3) pkg> add Flux



As an Amazon Associate I earn from qualifying purchases.

Julia language, read delimited files (CSV) with "readdlm"

 

When trying to execute:


wikiEVDraw = readdlm("wikipediaEVDraw.csv", ',') # getting quotes right is important!



I get the following error:

UndefVarError: readdlm not defined.


help?> readdlm search: readdir Couldn't find readdlm Perhaps you meant readdir, read, read!, real or readchomp No documentation found. Binding readdlm does not exist.


In this case use:

using DelimitedFileswikiEVDraw = readdlm("wikipediaEVDraw.csv", ',') # getting quotes right is important! 



As an Amazon Associate I earn from qualifying purchases.

Using Julia Language with AWS Lambda functions.

 

Here is a Julia language library to operate with AWS Lamda functions:

https://github.com/samoconnor/AWSLambda.jl#deploy-a-julia-lambda-function





As an Amazon Associate I earn from qualifying purchases.

Julia

The best way to start with Julia is to install JuliaPro, for free.





As an Amazon Associate I earn from qualifying purchases.

Arrays in Julia Language

 Way to create an Array in Julia language:


A simple way of creating an array


array18 = fill(18, 4) # values of 18, 4 rows


4-element Array{Int64,1}:
18
18
18
18




Float64, 2-dimensional,  4-rows, 3-columns

A = Array{Float64,2}(undef, 4,3) 

# Array of floats, 2-dimentional, populate with undefiled data, 4-rows, 3-columns


4×3 Array{Float64,2}:
2.40692e-314 5.0e-324 1.5e-323
2.39671e-314 2.40692e-314 1.5e-323
2.40692e-314 2.40692e-314 1.0e-323 
 2.40692e-314  2.40692e-314  2.38725e-314 


A[1,1] = 123

A




4×3 Array{Float64,2}:
123.0 5.0e-324 2.5e-323
2.38296e-314 2.38338e-314 2.5e-323
2.38338e-314 2.38338e-314 1.0e-323
2.38338e-314 2.38338e-314 2.64889e-32


Abstract Type Integer

A = Array{Integer}(undef,2,3)


2×3 Array{Integer,2}:
#undef #undef #undef
#undef #undef #undef


Concreate Type Int64

A = Array{Int64}(undef,2,3)

2×3 Array{Int64,2}:
4826207760 4826207824 4826207888
4826207792 4826207856 4825801360






Reference:



As an Amazon Associate I earn from qualifying purchases.

Special Characters used for Math, ML, Logic in Jupyther Lab


Here is a quick cheat sheet of useful Unicode characters you can use in Jupyther Lab and Julia language:


Greek Alphabet


α    \alpha
β    \beta
γ    \gamma
    \delta
ϵ    \epsilon
ζ    \zeta
η    \eta
θ    \theta
ι    \iota
κ    \kappa (should look like 'k')
λ    \lambda
μ    \mu
ν    nu (not supported in Jupyther)
ξ    \xi 
ο    omnicron (not supported in Jupyther)
π    \pi
ρ    \rho 
σ    \sigma
τ    \tau
υ    upsilon (not supported in Jupyther)
ϕ    \phi
χ    \chi
ψ    \psi
ω    \omega

Hebrew


ℵ    \aleph wiki

Machine Learning


    x\hat

Math

    \measuredangle
    \sum
±    \pm (plus minus)
    \ne
    \ge (greater or equal)
    \le (less or equal)
    \diameter
    0\degree 

Sets


    \in

Logic

    \therefore
    \xor 





As an Amazon Associate I earn from qualifying purchases.

Sublime





Download:


Terminus

SendCode




As an Amazon Associate I earn from qualifying purchases.

Using Linear Regression with Julia Language

Linear Regression


What is a Linear Regression?

Watch:

Selecting the best "fit" line to predict the data.

Fitting a linear regression model to a numerical data set.

Calculating the deviations (the error)


sum of squares






As an Amazon Associate I earn from qualifying purchases.

Boston


The Boston Housing Data Set


Download "boston.mat"


As an Amazon Associate I earn from qualifying purchases.

Juno

Read page:



Install Atom





As an Amazon Associate I earn from qualifying purchases.

pose

Body pose recognition has applications in:
  • image compression when only the position of the subject is needed
  • social signaling
  • ADAS driver monitoring system (DMS)
The human body pose consists of:
  • body detection in the image
  • body segmentation from the image
  • mapping the 2D image to the 3D model
  • tracking the pose over time (movie)





As an Amazon Associate I earn from qualifying purchases.

Whimsical Mind Graph - Explore Machine Learning

As I am studying Machine Learning I am creating the mind graph à la Daniel Bourke:

https://whimsical.com/K85bjCAHxKg4FZBf3YW7en




As an Amazon Associate I earn from qualifying purchases.

Julia Flux Meatalhead.jl




## Uncomment the lines below if you have not run it previously
# import Pkg; Pkg.add("Metalhead")
# import Pkg; Pkg.add("QuartzImageIO")

using Metalhead
using Metalhead: classify
vgg = VGG19()

## local file in this lab's folder
image = load("skunk001.jpeg")
classify(vgg, image)

output:


As an Amazon Associate I earn from qualifying purchases.

Conda nodejs




$ conda install -c conda-forge nodejs


As an Amazon Associate I earn from qualifying purchases.

Julia Flux







import Pkg; Pkg.add("Flux")
using Flux



As an Amazon Associate I earn from qualifying purchases.

Julia CUDA





import Pkg
Pkg.add("CUDAapi")




Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions... Installed CUDAapi ─ v4.0.0 Updating `~/.julia/environments/v1.4/Project.toml` [3895d2a7] + CUDAapi v4.0.0 Updating `~/.julia/environments/v1.4/Manifest.toml` [3895d2a7] + CUDAapi v4.0.0

using CUDAapi
has_CUDA = "false"
if has_cuda()
    @info "CUDA is on"
    has_CUDA = "true"
end
println("it is ", has_CUDA, " that I have CUDA.")



As an Amazon Associate I earn from qualifying purchases.

Heaven 12 1/2 design by Joel White - DIY part 1

 Episode 1 choosing the boat





















As an Amazon Associate I earn from qualifying purchases.

Fiberglass

 




As an Amazon Associate I earn from qualifying purchases.

bread

Natalia has baked a wonderful sourdough bread.

 




As an Amazon Associate I earn from qualifying purchases.

How much DNA do we share?

I was chatting with "likely" relative, geneticist, on my father's side.

- humans and chimps share 97-99%,
- humans, like myself, and Neanderthals share 3%,
- 3rd to 4th cousins share 0.8%, 

Should this explain why I feel more at home in the ZOO then over the birthday party of my grand-aunt? 

Written of course as a joke, still, it needs clarification. I hope to see an article soon.


As an Amazon Associate I earn from qualifying purchases.

My favorite quotations..


“A man should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.”  by Robert A. Heinlein

"We are but habits and memories we chose to carry along." ~ Uki D. Lucas


Popular Recent Articles