K5 Playground

Image analytics album with Google Cloud Storage and Vision API

60 min
Saito
  1. 1. Information
    1. 1.1. Ingredients
    2. 1.2. Functions
  2. 2. Create the application
    1. 2.1. Frontend
    2. 2.2. Backend
  3. 3. Customize the application
    1. 3.1. Frontend
    2. 3.2. Backend
  4. 4. Deploy the application
    1. 4.1. Backend
    2. 4.2. Frontend
  5. 5. Download
  6. 6. Version

Many of you have experience uploading photos to Facebook. And you notice that Facebook shows who is on the photos.

In this tutorial, I show you how to build this kind of function on your application. It’s complicated. But you’ll get it done in 60 minutes!

Information

Ingredients

  • Template: Web Camera
  • API Loigcs: Google Cloud Storage, Google Vision API

Functions

  • Snapshoting user camera.
  • Analyzing images by using Google Vision API.
    • Text detection
    • Face detection
    • Logo detection
    • Landmark detection
    • Label detection
    • Safe search detection
    • Image property
  • Storing analyzed images in Google Cloud Storage
  • Showing a list of the images.

Create the application

Frontend

  1. Choose the Web Camera Template.

  2. Customize the UI in K5 Playground.

Backend

  1. Create two WebAPIs

    1. Add the google_storage/files and activate GET method.

    2. Add the google_storage/files/{id} and activate DELETE method.

  2. Edit the API Logic of GET google_storage/files endpoint.

    1. Click GET google_storage/files endpoint.

    2. Add the Get File List API logic from Storage menu at the right pane.

  3. Edit the API Logic of DELETE google_storage/files/{id} endpoint.

    1. Click DELETE google_storage/files/{id} endpoint.

    2. Add the Delete Files API Logic from Storage menu at the right pane.

    3. Edit the code.

      Set the fileName to delete a file and return the response.
      The API should return the result of deleting a file.

      1
      2
      3
      4
      5
      var fileName = req.swagger.params.id.value;
      // ~~ code ommitted ~~
      bucket.file(fileName).delete(function(error, apiResponse) {
        next(apiResponse);
      });

  4. Edit the API Logic of POST /sample_annnotate_imagedata/image endpoint.

    1. Click POST /sample_annnotate_imagedata/image endpoint.

    2. Edit ‘Upload File’ API Logic.

      Set the file and the parameter of createWriteStream method for uploading a file to Google Storage.
      Note: Every file you upload will be public.

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      var file = req.swagger.params.file.value;
      // ~~ code ommitted ~~
      var remoteWriteStream = bucket.file(file.originalname)
        .createWriteStream({// Options (optional)
          gzip: false,      // Automatically gzip the file
          public: true,    // Make the uploaded file public
          private: false,    // Make the uploaded file private
          metadata: {       // metadata
            cacheControl: true,
            contentType: file.mimetype,
            metadata: {     // for custom metadata(key-value)
              description: 'file description' // example
            }
          }
        })

  5. Edit the API Logic of POST /sample_annnotate_imagedata/json endpoint.

    1. Click POST /sample_annnotate_imagedata/json endpoint.

  6. Set API Key to use Google Vision API.

    Set every apiKey of endpoints listed up below.

    • POST /sample_annnotate_imagedata/logo
    • POST /sample_annnotate_imagedata/safe_search
    • POST /sample_annnotate_imagedata/text
    • POST /sample_annnotate_imagedata/property
    • POST /sample_annnotate_imagedata/landmark
    • POST /sample_annnotate_imagedata/face
    • POST /sample_annnotate_imagedata/label

  7. Set the projectId, keyFileName and bucketName to use Google Storage.

    Set the projectId, keyFileName and bucketName of endpoints listed up below.

    • POST /sample_annnotate_imagedata/image
    • POST /sample_annnotate_imagedata/json
    • GET /google_storage/files
    • DELETE /google_storage/files/{id}

  8. Download the application.

Customize the application

Frontend

  1. Edit the constants/AppConstants.js.

    Add a new ActionType to handle a list of image stored in Google Storage.

    1
    2
    3
    4
    5
    6
    7
    const ActionTypes = {
      GET_ANNOTATE_IMAGE: 'GET_ANNOTATE_IMAGE',
      CLEAR_ANNOTATE_IMAGE: 'CLEAR_ANNOTATE_IMAGE',
      GET_ANALYZED_IMAGES: 'GET_ANALYZED_IMAGES',    // get a list of image stored in Google Storage
    };

    export default ActionTypes;

  2. Add a Store.

    Create a new Store named stores/AnalyzedImageStore.js to handle a list of images stored in Google Storage.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    import { ReduceStore } from 'flux/utils';
    import ActionTypes from '../constants/AppConstants';
    import AppDispatcher from '../dispatcher/AppDispatcher';

    class AnalyzedImageStore extends ReduceStore {
      getInitialState() {
        return {
          data: [],
        };
      }

      reduce(state, action) {
        switch (action.type) {
          case ActionTypes.GET_ANALYZED_IMAGES: {
            return {
              data: action.data,
            };
          }
          default: {
            return state;
          }
        }
      }
    }

    export default new AnalyzedImageStore(AppDispatcher);

  3. Edit the actions/AnnotateImageActionCreators.js.

    Add two methods to the action. getImages method retreives a list of files and deleteImage method deletes a file from Google Storage.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    const AnnotateImageActionCreators = {
    // ~~ code omitted ~~
      getImages({ resolve }) {
        api.get(
          '/google_storage/files'
        ).then((response) => {
          AppDispatcher.dispatch({
            type: ActionTypes.GET_ANALYZED_IMAGES,
            data: response.data.map(file => file.metadata),
          });
          resolve();
        }).catch(() => {
          // @todo handle error
        });
      },
      deleteImage(imageId) {
        api.delete(
          `/google_storage/files/${imageId}`
        ).then((response) => {
          this.getImages({
            resolve: () => {},
          });
        }).catch(() => {
          // @todo handle error
        });
      },

  4. Create a new page to show the album.

    1. Add a container component.

      Create a new component named components/container/AnalyzedImageAlbumContainer.js.

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      import React, { Component } from 'react';
      import { Container } from 'flux/utils';
      import AnalyzedImageAlbumPage from '../pages/AnalyzedImageAlbumPage';
      import AnalyzedImageStore from '../../stores/AnalyzedImageStore';

      class AnalyzedImageAlbumContainer extends Component {

        static getStores() {
          return [
            AnalyzedImageStore,
          ];
        }

        static calculateState() {
          return {
            analyzedImages: AnalyzedImageStore.getState(),
          };
        }

        render() {
          const {
            ...other
          } = this.props;

          return (
            <div className="report-container" style={{ height: '100%' }}>
              <AnalyzedImageAlbumPage
                images={this.state.analyzedImages.data}
                {...other}
              />
            </div>
          );
        }
      }

      export default Container.create(AnalyzedImageAlbumContainer);

    2. Add a page component.

      Create a new component named components/pages/AnalyzedImageAlbumPage.js.
      Note: This component uses react-markdown. You will install it at the deployment.

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      32
      33
      34
      35
      36
      37
      38
      39
      40
      41
      42
      43
      44
      45
      46
      47
      48
      49
      50
      51
      52
      53
      54
      55
      56
      57
      58
      59
      60
      61
      62
      63
      64
      65
      66
      67
      68
      69
      70
      71
      72
      73
      74
      75
      76
      77
      78
      79
      80
      81
      82
      83
      84
      85
      86
      87
      88
      89
      90
      91
      92
      93
      94
      95
      96
      97
      98
      99
      100
      101
      102
      103
      104
      105
      106
      107
      108
      109
      110
      111
      112
      113
      114
      115
      116
      117
      118
      119
      120
      121
      122
      123
      124
      125
      126
      127
      128
      129
      130
      131
      132
      133
      134
      import React, { Component, PropTypes } from 'react';
      import { Link } from 'react-router';
      import ReactMarkdown from 'react-markdown';
      import { ItemsWrapperResponsive003 } from 'material-ui-fj/sublayouts';
      import { Constants } from '../../constants/AppConstants';
      import AnnotateImageActionCreators from '../../actions/AnnotateImageActionCreators';
      import { VerticalCard003 } from 'material-ui-fj/Card';
      import { HorizontalLayout, VerticalLayout } from 'material-ui-fj/sublayouts';
      import { BorderedFlatButton003 } from 'material-ui-fj/Button';

      export default class AnalyzedImageAlbumPage extends Component {
        static propTypes = {
          images: PropTypes.arrayOf(
            PropTypes.object,
          ).isRequired,
        };

        static defaultProps = {};

        static contextTypes = {
          muiFjTheme: PropTypes.object.isRequired,
        };

        static getStyles(context) {
          const { palette } = context.muiFjTheme;
          const styles = {
            root: {
            },
            center: {
              maxWidth: 1200,
              margin: '24px auto',
            },
            itemsWrapperInner: {
              marginTop: 0,
            },
            cardContent: {
              paddingBottom: 12,
            },
            cardHighlighted: {
              fontSize: 18,
              fontWeight: 300,
              color: palette.primary1Color,
            },
            itemsFooter: {
              margin: '0 24px',
            },
            top: {
              overflow: 'hidden',
            },
          };
          return styles;
        }

        state = { }

        handleDeleteImage = (imageName) => {
          AnnotateImageActionCreators.deleteImage(imageName);
        }

        render() {
          const {
            images,
          } = this.props;

          const styles = AnalyzedImageAlbumPage.getStyles(this.context);

          const deleteButton = (imageName) => (
            <BorderedFlatButton003
              label="DELETE"
              onTouchTap={() => this.handleDeleteImage(imageName)}
            />
          );

          const downloadButton = (downloadUrl) => (
            <BorderedFlatButton003
              label="DOWNLOAD"
              href={downloadUrl}
            />
          );

          const card = value => {
            const imageDescription =
      `
      * size : ${value.size}
      * time created: ${value.timeCreated}
      `;

            const imageSrc = `https://storage.googleapis.com/${value.bucket}/${value.name}`;

            return (
              <VerticalLayout style={styles.container}>
                <VerticalCard003
                  key={value.id}
                  imageAlt={value.name}
                  imageSrc={imageSrc}
                  title={value.name}
                  subtitle=''
                  description={
                    <ReactMarkdown
                      className="react-markdown"
                      source={imageDescription}
                      escapeHtml
                    />
                  }
                  linkTo=''
                  contentStyle={styles.cardContent}
                  highlighted=""
                  highlightedStyle={styles.cardHighlighted}
                />
                <VerticalLayout>
                  <HorizontalLayout alignX={'center'}>
                    {deleteButton(value.name)}
                    {downloadButton(value.mediaLink)}
                  </HorizontalLayout>
                </VerticalLayout>
              </VerticalLayout>
            );
          };

          const cards = (
            images.map(value => card(value))
          );

          return (
            <div style={styles.center}>
              <ItemsWrapperResponsive003
                itemsWrapperInnerStyle={styles.itemsWrapperInner}
              >
                {cards}
              </ItemsWrapperResponsive003>
            </div>
          );
        }
      }

    3. Add the album page to the menu by editting components/pages/WebCameraPages.js.

      1. Add the import statement at the top of the file.
      1
      2
      import AnalyzedImageAlbumContainer from '../containers/AnalyzedImageAlbumContainer';
      import Divider from 'material-ui/Divider';
      1. Add the AnalyzedImageAlbumContainer component to const contents.
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      const contents = [
        <AnnotateImageContainer type="FACE_DETECTION" {...other} />,
        <AnnotateImageContainer type="LABEL_DETECTION" {...other} />,
        <AnnotateImageContainer type="TEXT_DETECTION" {...other} />,
        <AnnotateImageContainer type="LANDMARK_DETECTION" {...other} />,
        <AnnotateImageContainer type="LOGO_DETECTION" {...other} />,
        <AnnotateImageContainer type="SAFE_SEARCH_DETECTION" {...other} />,
        <AnnotateImageContainer type="IMAGE_PROPERTIES" {...other} />,
        <div />,
        <AnalyzedImageAlbumContainer {...other} />,
      ];

      1. Add a new MenuItem component to show the page menu.

        1
        2
        3
        4
        5
        6
        7
        8
            const left = (
        // ~~ code omitted ~~
                      <MenuItem value={4} primaryText="Logo" />
                      <MenuItem value={5} primaryText="Safe Search" />
                      <MenuItem value={6} primaryText="Image Properties" />
                      <Divider />
                      <MenuItem value={8} primaryText="Album" />
                    </SidebarMenu>

      2. Modify handleClick method.

      Before showing the album page, the application needs to reflesh image list.

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      handleClick = (event, menuItem, index) => {
        AnnotateImageActionCreators.clear();
        if (index === 8) {
          // Reflesh image list
          AnnotateImageActionCreators.getImages({
            resolve: () => this.setState({ displayPage: index }),
          });
        } else {
          this.setState({ displayPage: index });
        }
      }

Backend

Nothing to be edited.

Deploy the application

Finally, all you need to do is deploying backend and frontend.

Backend

1
2
cd backend
cf push [APP_NAME]

Frontend

1
2
3
4
5
6
7
cd frontend
npm install
npm install --save react-markdown
set API_URL=[Backend URL] # for Windows
export API_URL=[Backend URL] # for Mac/Linux
npm run build
cf push [APP_NAME] -p public

Download

Version

  • Web Camera Template: v1.0.0
Tutorial React.js Flux Node.js Swagger BFF SPA Microservice K5 K5 CF WebCamera Google