Missing xcrun on macOS High Sierra


Error

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

error: "" unsupported

Solution:


  • Re-install XCode from App Store (~20 minutes)
    • Open XCode and agree to lice agreement
    • Allow it to "Instal components..."
This resolved my problem, but optionally you can try the following:
  • xcode-select --install
  • sudo xcodebuild -license
  • xcode-select --reset







As an Amazon Associate I earn from qualifying purchases.

sed command line tool

The sed command line tool allows you to pipe a string of text and substitute part of it.

Note that it does for the first occurrence only:



$ echo "my, this is my sentence" | sed 's/my/My/'
My, this is my sentence


Escaping forward slashes with the backslashes:


$ echo "convert /usr/local/bin to /common/bin" | sed "s/\/usr\/local\/bin/\/common\/bin/"
convert /common/bin to /common/bin



$ echo "Repeat me 5 times." | sed 's/[0-9]/& & & & &/'

Repeat me 5 5 5 5 5 times.



As an Amazon Associate I earn from qualifying purchases.

How to change Python version (environment) using Anaconda?

$ python --version

Python 3.6.1 :: Anaconda 4.4.0 (x86_64)

$ conda create -n py27 python=2.7 anaconda

$ source activate py27

$ python --version

Python 2.7.13 :: Anaconda 4.4.0 (x86_64)


Also posted on GitHub


As an Amazon Associate I earn from qualifying purchases.

How to show hidden files in Apple OS X Finder (AppleShowAllFiles)


Open Terminal and execute the following line:

$ defaults write com.apple.finder AppleShowAllFiles YES


As an Amazon Associate I earn from qualifying purchases.

Python CAN bus

Install Python CAN library


~ uki$ sudo pip install python-can
Collecting python-can
Downloading python-can-1.5.2.tar.gz (77kB)
100% |████████████████████████████████| 81kB 3.3MB/s
Installing collected packages: python-can
Running setup.py install for python-can ... done
Successfully installed python-can-1.5.2

Follow the Directions


https://python-can.readthedocs.io/en/latest/installation.html


As an Amazon Associate I earn from qualifying purchases.

CyberFit

Last year, I signed up with a gym trainer. 

Over a few weeks, I observed how he tracked my progress on the tablet with spreadsheets. 
That included goals, exercise history, and measurements. We took short videos of proper techniques. 

The idea of the app was obvious. Each trainer should be able to create a program, post videos, keep track of the client's progress, and make money. Training buddies should be able to share their progress and schedules. The app creation process was a great motivation to stay fit, too!







As an Amazon Associate I earn from qualifying purchases.

Android Adaptive Icons

Android allows us to create adaptive icons (various shapes for one icon for different devices) for:
  • launcher icons
  • shortcuts
  • Setting app
  • sharing dialogs
  • overview screen
You have to provide the following layers 108x108 dp:
  • top mask (circle, squircle, rounded square, square) provided by device OEM (e.g. Samsung)
  • foreground layer 72x72 dp should be visible in the mask viewport
  • background layer

The edge 18 dp are reserved for system's special effects.

Use Android Studio 3.0 Asset Studio to create these adaptive icons.



Reference:

https://developer.android.com/preview/features/adaptive-icons.html


As an Amazon Associate I earn from qualifying purchases.

Python UnicodeDecodeError



When you are reading CSV file (spreadsheet), make sure you set encoding="utf-8", or you might get:

UnicodeDecodeError: 'utf-8' codec can't decode byte
(result, consumed) = self._buffer_decode(data, self.errors, final)



def read_rows(CSV_file_path: str):
 
    import sys
    print("default encoding", sys.getdefaultencoding())
    import csv
    rows = []
    with open(CSV_file_path, 'rt', encoding="utf-8") as csvfile:
        cvs_reader = csv.reader(csvfile, delimiter=',', quotechar='"')
        for row in cvs_reader: # row is a list
            rows.append(row)
    return rows


As an Amazon Associate I earn from qualifying purchases.

Using Docker containers on Mac

Install Docker on Mac


https://download.docker.com/mac/stable/Docker.dmg


$ docker --version
Docker version 17.03.1-ce, build c6d412e

$ docker-compose --version
docker-compose version 1.11.2, build dfed245

$docker-machine --version
docker-machine version 0.10.0, build 76ed2a6

Docker for Udacity Machine Learning

$ docker pull kyoungrok0517/tensorflow-udacity

Docker for Raspberry Pi

Having created a large Pi cluster, the next logical step is to deploy various microservices in Docker containers on Pi.

https://www.raspberrypi.org/blog/docker-comes-to-raspberry-pi/












As an Amazon Associate I earn from qualifying purchases.

Python OCR for robotics.

I was thinking that it would be cool for my robots to have text reading capability, also known as OCR.

I was testing Python pytesseract library:


https://github.com/UkiDLucas/uki.guru/blob/master/OCR/ocr.ipynb


So far no success, but it will be worth trying.


As an Amazon Associate I earn from qualifying purchases.

Herreshoff

 




As an Amazon Associate I earn from qualifying purchases.

Python: Passing *args and **kwargs into a function

# *args is the usual notation for optional, non-keyword arguments
# **kwargs is the usual notation for optional, keyword arguments
def test_var_args(regular_argument, *args, **kwargs):
    print ("- regular argument:", regular_argument)
    for arg in args:
        print ("- unnamed arguments arg:", arg)
    for key in kwargs:
        print ("- named arguments: %s: %s" % (key, kwargs[key]))
test_var_args(1, "two", 3, four=4, five="five")


- regular argument: 1
- unnamed arguments arg: two
- unnamed arguments arg: 3
- named arguments: four: 4
- named arguments: five: five


As an Amazon Associate I earn from qualifying purchases.

Python bubble sort

Since I had to learn python for my machine learning and self-driving classes and I had to review the basics. Not to waste my efforts, I am creating a new guide, you can see more in:

https://github.com/UkiDLucas/python_zen_interviews

def bubble_sort(the_list: list, verbose: int=0):
    """
    author: @UkiDLcuas
    This function changes the provided list.
    The list can contain integers, decimal numbers, strings, etc.
    The list is sorted in ascending order (first to last).
    The function does not return anything.
    """
    if verbose > 0:
        iteration = 0
   
    # count remining bubbles ( aka step backwards)
    start = len(the_list)-1 # end, zero-based list
    stop = 0 # beginning
    step = -1 # backwards
    for remaining_bubbles in range(start, stop, step):
        for i in range(remaining_bubbles):
            if verbose > 0:
                iteration = iteration + 1
                print("iteration", iteration, "remaining_bubbles", remaining_bubbles, "index", i)
                print("  ", the_list)
                print("    comparing if is", the_list[i], "bigger than", the_list[i+1])
            if the_list[i] > the_list[i+1]:
                # swap
                temp = the_list[i+1] # temp placehoder for the value to be moved
                the_list[i+1] = the_list[i] # bubble up
                the_list[i] = temp # bubble down
    if verbose > 0:
        print("*** finished", len(the_list), "element list in ", iteration, "iterations")


As an Amazon Associate I earn from qualifying purchases.

Sailing on Amazon river? - No, not really.

This video is not related to sailing on great lakes, but it is amazing and worth seeing...



As an Amazon Associate I earn from qualifying purchases.

AWS

Notes on how I use Amazon AWS EC2 instances for Convolutional Deep Neural Networks Machine Learning, using powerful GPU CUDA configurations.

I have moved this post to:

https://ukidlucas.github.io/posts/AWS.html


As an Amazon Associate I earn from qualifying purchases.

Ubuntu: installing TensorFlow for NVidia (CUDA) GPU


I am setting TensorFlow on:
  • Ubuntu 16.04 LTS 64-bit
  • 16 GiB RAM
  • AMD Athlon(tm) II X4 640 Processor × 4 
  • GeForce GTX 1050 Ti/PCIe/SSE2

Check if you have NVidia CUDA GPU

uki@uki-p6710f:~$  lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 1c82 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 0fb9 (rev a1)

Check the name of your OS

uki@uki-p6710f:~$ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Check C compiler

uki@uki-p6710f:~$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Ubuntu Headers 


uki@uki-p6710f:~$ sudo apt-get install linux-headers-$(uname -r)
[sudo] password for uki:
Reading package lists... Done
Building dependency tree
Reading state information... Done
linux-headers-4.4.0-62-generic is already the newest version (4.4.0-62.83).
linux-headers-4.4.0-62-generic set to manually installed.
The following packages were automatically installed and are no longer required:
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic linux-image-extra-4.4.0-31-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 89 not upgraded.

Download newest CUDA installer (1.4GB)


https://developer.nvidia.com/cuda-downloads

https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda_8.0.61_375.26_linux-run

Execute CUDA installer

 cd ~/Downloads/
uki@uki-p6710f:~/Downloads$ ls -alt

Mar  5 14:54 cuda_8.0.61_375.26_linux.run
Feb  2 09:53 NVIDIA-Linux-x86_64-375.10.run

$ sudo sh cuda_8.0.61_375.26_linux.run

Set environment variables

uki@uki-p6710f:~$ nano ~/.bashrc

##### CUDA 
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64



uki@uki-p6710f:~$ echo $LD_LIBRARY_PATH

/usr/local/cuda-8.0/lib64

Install TensorFlow via pip3 (Python 3.5)


uki@uki-p6710f:~$ python --version
Python 3.5.2 :: Anaconda 4.3.0 (64-bit)

$ sudo apt install python3-pip


$  conda info --envs
# conda environments:
#
tensorflow               /home/uki/anaconda3/envs/tensorflow
root                  *  /home/uki/anaconda3


$ conda env create -f /Users/ukilucas/dev/uki.guru/conda_enviroment_GPU.yml

$source activate tensorflow


Setting Jupyter kernel to match Python conda environment


http://ukitech.blogspot.com/2017/02/kernel.html



As an Amazon Associate I earn from qualifying purchases.

Removing unused Jupyter notebook kernels on Mac

Since I am experimenting with various Python versions 2 vs 3.5 vs 3.6 and packages, e.g. TensorFlow I often end up with conda environments and corresponding Jupyter kernels that I do not use.

Removing Kernels


$ ls -alt ~/Library/Jupyter/kernels/
total 0
drwx------  6 uki  staff  192 Oct 23 07:45 ../
drwxr-xr-x  5 uki  staff  160 Oct 23 07:43 ./
drwxr-xr-x  5 uki  staff  160 Oct 23 07:43 py27/
drwxr-xr-x  5 uki  staff  160 Sep 30 19:59 python2/

drwxr-xr-x  3 uki  staff   96 Jan 23  2018 bash/

$ rm -r ~/Library/Jupyter/kernels/python2/

See my post on:

Installing Jupyter kernels to match Conda environments


https://ukidlucas.blogspot.com/2018/10/ipykernel.html


As an Amazon Associate I earn from qualifying purchases.

Developing Alexa skills using flask-ask Python framework

Install Flask-Ask for Alexa


$ pip install flask-ask

Start your favorite Python dev editor


$ jupyter notebook

Write code


To avoid repeating myself in 2 places, here is my code on GitHub:

https://raw.githubusercontent.com/UkiDLucas/uki.guru/master/Alexa/skill_001.py

https://github.com/UkiDLucas/uki.guru/blob/master/Alexa/templates.yaml


Test code running locally via HTTPS using ngrok


$ python skill_001.py  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 275-052-815





https://ngrok.com/

- unzip
- put in Applications folder
- click to open
- you should get a message "ngrok was blocked from opening because it is not from an identified developer"
- got to System Preferences -> Security & Privacy
- click "Open Anyway"
- run from Terminal:



$ /Applications/ngrok http 5000

You will need the HTTPS URL form ngrok


ngrok by @inconshreveable                                                                                                                                                (Ctrl+C to quit)                                                                                                                                                                                         Session Status                online                                                                                                                                                     Version                       2.1.18                                                                                                                                                     Region                        United States (us)                                                                                                                                         Web Interface                 http://127.0.0.1:4040                                                                                                                                      Forwarding                    http://908cd95a.ngrok.io -> localhost:5000                                                                                                                 Forwarding                    https://908cd95a.ngrok.io -> localhost:5000                                                                                                                                                                                                                                                                                                         Connections                   ttl     opn     rt1     rt5     p50     p90                                                                                                                                              0       0       0.00    0.00    0.00    0.00 


Log in to Amazon Apps and Services


Make sure you use the same email and you have set up your Alexa Echo device with.

https://developer.amazon.com/edw/home.html#/


Alexa Skills Kit
Easily add new skills to Alexa






As an Amazon Associate I earn from qualifying purchases.

Installing TensorFlow 1.0 as conda environment with yml definition file on MacBook Pro with NVidia GPU support



My conda environment yml file on GitHub

https://github.com/UkiDLucas/uki.guru/blob/master/conda_enviroment_GPU.yml

Execute yml file, set environment




$ conda env create --file .../conda_environment.yml
$ conda info --envs  
$ source activate py352_tf_gpu




Update jupyter notebook (iPython) kernel with new environment we created



$ python -m ipykernel install --user --name py352_tf_gpu --display-name "conda env py352_tf_gpu"$ jupyter notebook

Test run in jupyter notebook


You can see that jupyter notebook recognize GPU




You can see execution on MacBook Pro (late 2013 with NVidia). Both CPU and GPU are measured.





As an Amazon Associate I earn from qualifying purchases.

Google TensorFlow 1.0 - Mac with Nvidia GPU installation and testing in jupyter notebook


TensorFlow 1.0 is promising more stable GPU build for Mac, let's test it.


Make sure you have CUDA installed


$ echo $DYLD_LIBRARY_PATH/usr/local/cuda/lib:


http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/#axzz4Ynrb42hi



$ echo $CUDA_HOME/Developer/NVIDIA/CUDA-8.0/

https://developer.nvidia.com/cudnn


$ sudo easy_install pip

Set variable with TensorFlow 1.0 binary



export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl
$ echo $TF_BINARY_URL https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl



Use pip to install TensorFlow




$ sudo pip3 install --upgrade  $TF_BINARY_URL Password:The directory '/Users/ukilucas/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.The directory '/Users/ukilucas/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.Collecting tensorflow-gpu==1.0.0 from https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl  Downloading https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl (89.0MB)    100% |████████████████████████████████| 89.0MB 8.8kB/s Requirement already up-to-date: numpy>=1.11.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: wheel>=0.26 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: six>=1.10.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: protobuf>=3.1.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Collecting setuptools (from protobuf>=3.1.0->tensorflow-gpu==1.0.0)  Downloading setuptools-34.2.0-py2.py3-none-any.whl (389kB)    100% |████████████████████████████████| 399kB 2.0MB/s Requirement already up-to-date: appdirs>=1.4.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: packaging>=16.8 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: pyparsing in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Installing collected packages: tensorflow-gpu, setuptools  Found existing installation: tensorflow-gpu 0.12.1    Uninstalling tensorflow-gpu-0.12.1:      Successfully uninstalled tensorflow-gpu-0.12.1  Found existing installation: setuptools 34.1.1    Uninstalling setuptools-34.1.1:      Successfully uninstalled setuptools-34.1.1Successfully installed setuptools-34.2.0 tensorflow-gpu-1.0.0(tensorflow_gpu) uki@UkiPEsMcBookPro 192.168.1.24 19:44 dev $ 



Start jupyter notebook in the right (same) environment



(tensorflow_gpu) $ jupyter notebook

Try in jupyter notebook


import tensorflow as tf
import time
from tensorflow.python.client import device_lib

def get_available_CPU_GPU():
   devices = device_lib.list_local_devices()
   #return [x.name for x in devices if x.device_type == 'CPU']
   return [x.name for x in devices ]

print(get_available_CPU_GPU())
['/cpu:0', '/gpu:0']


start = timeit.timeit()
print ("starting")
with tf.device('/gpu:0'):
    # [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print (sess.run(c))
end = timeit.timeit()
print ("elapsed", end - start)
starting
[[ 22. 28.]
[ 49. 64.]]
elapsed -0.003607149003073573

It is a win so far, time will show if it is usable.

Reference:

https://www.tensorflow.org/install/install_mac


As an Amazon Associate I earn from qualifying purchases.

Nvidia just released an update to their CUDA drivers for Mac

Nvidia CUDA just released an update to their Mac OS X driver.



As an Amazon Associate I earn from qualifying purchases.

Corrupt NVidia driver causes user to be stuck in the login screen

I was playing with dual monitors on my NVidia GTX 1050Ti card and at some point, the UI stopped to allow me to log in. It was going in the loop after each (correct) login.

Steps to fix it:

Press Ctrl+Alt+F1
or F2, F3

Log into in Command Line

Remove NVidia drivers
apt-get purge nvidia-*
Add repository
add-apt-repository ppa:graphics-drivers/ppa 

Refresh packages
sudo apt-get update


Install your NVidia driver  GTX 1050Ti driver 275.10
apt-get install nvidia-375

Reboot

sudo reboot

This has fixed it for me!


As an Amazon Associate I earn from qualifying purchases.

Python: converting PIL.Image to nympy array and back, resizing image

The example, I have used today:
import PIL
import numpy
from PIL import Image

def resize_image(numpy_array_image, new_height):
# convert nympy array image to PIL.Image
image
= Image.fromarray(numpy.uint8(numpy_array_image))
old_width
= float(image.size[0])
old_height
= float(image.size[1])
ratio
= float( new_height / old_height)
new_width
= int(old_width * ratio)
image
= image.resize((new_width, new_height), PIL.Image.ANTIALIAS)
# convert PIL.Image into nympy array back again
return array(image)




As an Amazon Associate I earn from qualifying purchases.

Installing Keras (with TensorFlow and Thano) using Python conda





(tensorflow_cpu) $ anaconda search -t conda keras | grep forge

     conda-forge/keras         |    1.0.7 | conda           | linux-64, win-32, win-64, osx-64

(tensorflow_cpu) $ conda install -c conda-forge keras=1.0.7


As an Amazon Associate I earn from qualifying purchases.

Cloning conda environments



You can clone (duplicate, copy) environments, this is useful when you are experimenting with packages whereas everything else is working fine:

- Existing tensorflow_gpu
- Newly created tensorflow_cpu


$ conda create --name py37 --clone base

List your environments :
$ conda info --envs
# conda environments:
#
                         /Users/uki/.julia/conda/3
                         /Users/uki/.julia/packages/ORCA/uEiWT/deps
base                  *  /anaconda3
py2                      /anaconda3/envs/py2
py37                     /anaconda3/envs/py37

Change to the newly created one:

$ source activate py37
Register this environment with Jupyter Notebook


$ python3 -m ipykernel install --user --name py37 --display-name "Python 3.7 (py37)"


Installed kernelspec py37 in /Users/uki/Library/Jupyter/kernels/py37


$ cd /Users/uki/Library/Jupyter/kernels/py37
$ code kernel.json

{
"argv": [
"/anaconda3/bin/python3",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"display_name": "Python 3.7 (py37)",
"language": "python"
}





As an Amazon Associate I earn from qualifying purchases.

Setting Jupyter kernel to match Python conda environment


For Jupyter kernel to reflect changes in conda env (tensorflow), I had to set up the following:




$ source activate tensorflow
$ sudo pip install ipykernel
$ python -m ipykernel install --user --name tensorflow --display-name "conda env tensorflow"



As an Amazon Associate I earn from qualifying purchases.

Installing TensorFlow GPU on MacBook Pro (late 2013)

Note, installation works, but GPU TensorFlow does not.




CPU VERSION (tensorflow 0.12.1):




import tensorflow as tf
print("OUT: tensorflow imported")


OUT: tensorflow imported


from tensorflow.python.client import device_lib
def get_available_CPU_GPU():
    devices = device_lib.list_local_devices()
    #return [x.name for x in devices if x.device_type == 'CPU']
    return [x.name for x in devices ]
print(get_available_CPU_GPU())


['/cpu:0']


SAME CODE GPU VERSION (tensorflow-gpu 0.12.1 ):


OUT: tensorflow imported


print(get_available_CPU_GPU())

---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
in ()
----> 1 from tensorflow.python.client import device_lib
2
3 def get_available_CPU_GPU():
4 devices = device_lib.list_local_devices()
5 #return [x.name for x in devices if x.device_type == 'CPU']

ImportError: No module named 'tensorflow.python'



I have logged the issue:
https://github.com/tensorflow/tensorflow/issues/7193


=====

Check for newest version:
TensorFlow get started page

I previously did export TF_BINARY_URL=..

$ echo $TF_BINARY_URL

https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-0.12.1-py3-none-any.whl

$ sudo  pip3 install --ignore-installed --upgrade $TF_BINARY_URL

The directory '/Users/ukilucas/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/ukilucas/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting tensorflow-gpu==0.12.1 from https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-0.12.1-py3-none-any.whl
  Downloading https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-0.12.1-py3-none-any.whl (83.6MB)
    100% |████████████████████████████████| 83.6MB 8.5kB/s 
Collecting numpy>=1.11.0 (from tensorflow-gpu==0.12.1)
  Downloading numpy-1.12.0-cp35-cp35m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.4MB)
    100% |████████████████████████████████| 4.4MB 165kB/s 
Collecting protobuf>=3.1.0 (from tensorflow-gpu==0.12.1)
  Downloading protobuf-3.2.0-py2.py3-none-any.whl (360kB)
    100% |████████████████████████████████| 368kB 1.2MB/s 
Collecting six>=1.10.0 (from tensorflow-gpu==0.12.1)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting wheel>=0.26 (from tensorflow-gpu==0.12.1)
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 2.9MB/s 
Collecting setuptools (from protobuf>=3.1.0->tensorflow-gpu==0.12.1)
  Downloading setuptools-34.1.0-py2.py3-none-any.whl (389kB)
    100% |████████████████████████████████| 399kB 1.1MB/s 
Collecting appdirs>=1.4.0 (from setuptools->protobuf>=3.1.0->tensorflow-gpu==0.12.1)
  Downloading appdirs-1.4.0-py2.py3-none-any.whl
Collecting packaging>=16.8 (from setuptools->protobuf>=3.1.0->tensorflow-gpu==0.12.1)
  Downloading packaging-16.8-py2.py3-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow-gpu==0.12.1)
  Downloading pyparsing-2.1.10-py2.py3-none-any.whl (56kB)
    100% |██████████████████    100% |██████████████████    100% |██████████████████    100% |████████████████████████████████| 61kB 2.0MB/s 
Installing collected packages: numpy, six, appdirs, pyparsing, packaging, setuptools, protobuf, wheel, tensorflow-gpu
Successfully installed appdirs-1.4.0 numpy-1.12.0 packaging-16.8 protobuf-3.2.0 pyparsing-2.1.10 setuptools-34.1.0 six-1.10.0 tensorflow-gpu-0.12.1 wheel-0.29.0




(tensorflow) $ conda list
# packages in environment at /Users/ukilucas/anaconda3/envs/tensorflow:
#
alabaster                 0.7.9                    py35_0  
appdirs                   1.4.0                    
appnope                   0.1.0                    py35_0  
appscript                 1.0.1                    py35_0  
numpy                     1.12.0                   
openssl                   1.0.2k                        0  
packaging                 16.8                     
pip                       9.0.1                    py35_1  
protobuf                  3.2.0                    
pyparsing                 2.1.10                   
python                    3.5.2                         0  
readline                  6.2                           2  
setuptools                27.2.0                   py35_0  
setuptools                34.1.0                   
six                       1.10.0                   
sqlite                    3.13.0                        0  
tensorflow-gpu            0.12.1                   
tk                        8.5.18                        0  
wheel                     0.29.0                   py35_0  
wheel                     0.29.0                   
xz                        5.2.2                         1  

zlib                      1.2.8                         3  


As an Amazon Associate I earn from qualifying purchases.

Python import error for sklearn.model_selection 0.17.1

ImportError: No module named 'sklearn.model_selection'

Before doing the embarrassing things I did below, read this:
Setting Jupyter kernel to match Python conda environment


Change to conda environment you need to update (tensorflow_gpu in may case).

List packages that have "scikit" in them.

(tensorflow_gpu) $ conda list | grep scikit


Nothing, nada.


$ conda install scikit-learn
Fetching package metadata .........
Solving package specifications: .
Package plan for installation in environment /Users/ukilucas/anaconda3/envs/tensorflow_gpu:
The following NEW packages will be INSTALLED:
    mkl:          2017.0.1-0            numpy:        1.11.3-py35_0         scikit-learn: 0.18.1-np111py35_1    scipy:        0.18.1-np111py35_1




(tensorflow_gpu)  $ conda list | grep scikit

scikit-learn              0.18.1              np111py35_1 



import sklearn
print (sklearn.__version__)
from sklearn.model_selection import train_test_split
0.18.1

You do not have to re-start Jupyter Notebook
that is all!

And here is my very long and embarrassing story:


Check the version of the package

import sklearn
print (sklearn.__version__)
0.17.1

Install the latest package

$ conda install scikit-learn

installs:
scikit-learn-0.18.1

However, already running jupyter notebook does not see the change.

Making sure I am in the right  (tensorflow) environment:

$ source activate tensorflow
$ conda install scikit-learn


installs:

    mkl:          2017.0.1-0        
    numpy:        1.11.3-py35_0     
    scikit-learn: 0.18.1-np111py35_1
    scipy:        0.18.1-np111py35_1



Still, Jupyter Notebook does not see scikit-learn 0.18.1

Restarting jupyter notebook from the same tensorflow environment


(tensorflow) uki@Uki ~ $ jupyter notebook ~/dev/

Still, Jupyter Notebook does NOT see scikit-learn 0.18.1



(tensorflow) uki@Uki ~ $ conda update --all
Fetching package metadata .......
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at /Users/ukilucas/anaconda3/envs/tensorflow:
#
mkl                       2017.0.1                      0  
numpy                     1.11.3                   py35_0  
openssl                   1.0.2k                        0  
pip                       9.0.1                    py35_1  
python                    3.5.2                         0  
readline                  6.2                           2  
scikit-learn              0.18.1              np111py35_1  
scipy                     0.18.1              np111py35_1  
setuptools                27.2.0                   py35_0  
sqlite                    3.13.0                        0  
tk                        8.5.18                        0  
wheel                     0.29.0                   py35_0  
xz                        5.2.2                         1  
zlib                      1.2.8                         3  

(tensorflow) uki@UkiPEsMcBookPro 192.168.1.77 09:03 ~ $ 


Jupyter Kernels

Ok, so there is a clear distinction of the conda env (in my case tensorflow) from which I am starting Jupyter Notebook and what the kernels see.

I tried to switch kernels Python [conda root] and Python [default], NO difference: sklearn.__version__ = 0.17.1


(py35) ~ $ conda list
# packages in environment at /Users/ukilucas/anaconda3/envs/py35:
#
cycler                    0.10.0                   py35_0  
freetype                  2.5.5                         2  
icu                       54.1                          0  
libpng                    1.6.27                        0  
matplotlib                2.0.0               np111py35_0  
mkl                       2017.0.1                      0  
mock                      2.0.0                    py35_0  
numpy                     1.11.3                   py35_0  
olefile                   0.44                     
openssl                   1.0.2k                        0  
pandas                    0.19.2              np111py35_1  
pbr                       1.10.0                   py35_0  
Pillow                    4.0.0                    
pip                       9.0.1                    py35_1  
protobuf                  3.1.0                    py35_0    conda-forge
pyparsing                 2.1.4                    py35_0  
pyqt                      5.6.0                    py35_2  
python                    3.5.2                         0  
python-dateutil           2.6.0                    py35_0  
pytz                      2016.10                  py35_0  
qt                        5.6.2                         0  
readline                  6.2                           2  
scikit-learn              0.18.1                   
scikit-learn              0.18.1              np111py35_1  
scipy                     0.18.1              np111py35_1  
seaborn                   0.7.1                    py35_0  
setuptools                27.2.0                   py35_0  
sip                       4.18                     py35_0  
six                       1.10.0                   py35_0  
sklearn                   0.0                      
sqlite                    3.13.0                        0  
tensorflow                0.12.1                   py35_1    conda-forge
tk                        8.5.18                        0  
wheel                     0.29.0                   py35_0  
xz                        5.2.2                         1  
zlib                      1.2.8                         3  

(py35) ~ $ jupyter notebook ~/dev/

NO difference: sklearn.__version__ = 0.17.1
Jupyter Notebook does NOT see scikit-learn 0.18.1



Removing Old Packages




$ conda listscikit-learn 0.18.1
sklearn 0.0
(py35) ~ $ conda remove scikit-learn
(py35) ~ $ pip uninstall scikit-learn
(py35) ~ $ pip uninstall sklearn
(py35) ~ $ conda install scikit-learn
(py35) ~ $ jupyter notebook ~/dev/


Did not help!


Check which Python Executable is being used


import sys

sys.executable
'/Users/ukilucas/anaconda3/bin/python'


$ conda list


# packages in environment at /Users/ukilucas/anaconda3/envs/py35:




Deleting Old and Reinstalling Jupyter Notebook



from jupyter_core.paths import jupyter_data_dir 


print(jupyter_data_dir())


/Users/ukilucas/Library/Jupyter
Deleting :
- /Users/ukilucas/Library/Jupyter
- ~/.ipython
- ~/.jupyter
(py35)  ~ $ jupyter --version
4.2.0

(py35)  ~ $ jupyter notebook --generate-config
Writing default config to: /Users/ukilucas/.jupyter/jupyter_notebook_config.py
(py35)  dev $ jupyter notebook
[I 09:52:59.160 NotebookApp] [nb_conda_kernels] enabled, 2 kernels found



Did not HELP!!



Removing Anaconda3 and Jupyter - fresh installs

$ rm -r ~/Library/Jupyter
$ rm -r ~/.ipython
$ rm -r ~/.jupyter
$ rm -r ~/anaconda3/


Installation (Mac OSX)


Downloads $ wget https://repo.continuum.io/archive/Anaconda3-4.3.0-MacOSX-x86_64.sh

Note the new Python is 3.6, but for TensorFlow I will need 3.5

bash Anaconda3-4.3.0-MacOSX-x86_64.sh


Anaconda3 will now be installed into this location:

/Users/ukilucas/anaconda3

Refresh Bash Terminal

$ source ~/.bash_profile
conda info --envs

root                  *  /Users/ukilucas/anaconda3
conda list
...
python                    3.6.0 
...


Create a new enviroment for Python 3.5

TensorFlow Installation instructions
# Python 3.5
$ conda create
-n tensorflow python=3.5


At this time GPU Mac version is available only via pip


# Mac OS X, GPU enabled, Python 3.4 or 3.5:
(tensorflow)$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-0.12.1-py3-none-any.whl

$ sudo  pip3 install --ignore-installed --upgrade $TF_BINARY_URL

Summary

Full delete of Anaconda3 and Jupyter and fresh re-installation fixed the problem.




As an Amazon Associate I earn from qualifying purchases.

My favorite quotations..


“A man should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.”  by Robert A. Heinlein

"We are but habits and memories we chose to carry along." ~ Uki D. Lucas


Popular Recent Articles