What is the difference between the hierarchical and Non-Hierarchical Taxonomies?

When I was using a plugin to create #custom post type and #custom taxonomy, I came across following situation.

After creating custom taxonomy, I was hoping for was a simple option where I could uses the check box UI for selecting categories on each post.

Instead of that I got comma-separated text input.

I did some research and found out that, that plugin use a setting “Hierarchical ==>> false” when registering custom taxonomy.
‘hierarchical’=>false you get the metabox   format WordPress uses for Post Tags:

‘hierarchical’=>true you get the meta box format that WordPress uses for Categories: