Cross Compiling FFmpeg with x264 for Android

Recently we were working on a feature where we had to combine an image and audio to create a video on mobile devices. In iOS this can be done using AVAssetExportSessionthough – for detail see this link However, we could not find any native solution for this problem in Android.

FFmpeg is one such tool to tackle this problem but it is not available for Android officially. We tried few existing Android ports of FFmpeg but they were either outdated or didn’t work for us. So we planned to fold our sleeves and compile the library for Android.

Compiling libraries on Linux system is a fairly common task, download the source code of the library and run three commands:

./configure 
make 
make install

As we wanted to use the FFmpeg library on Android devices, we can’t just use the executable generated on the Linux machine directly because Android devices have different CPU architecture, different instruction sets and modified Linux kernel (OS). So we needed to cross compile FFmpeg library for Android

Cross Compilation ?

The process of building executable binaries on one machine, and running them on another machine when the CPU architecture or the Operating System are different is called “cross compilation”.

Continue reading

 

Vinsol Introduces ShopSpree: First iOS App for Spree Commerce

We understand the preferences of shoppers have changed globally. Today, consumers prefer shopping in the most convenient ways: anywhere, anytime and on the move. This changing landscape has given birth to the world of mobile and applications. The statistics prove the increase in mobile internet usage is on the rise, now standing at 65% globally.

ShopSpree0

Around 32% of all online purchases have been done through mobile. Mobile commerce is looking promising, and expects the spending to be $200 billion USD by the end of 2014, reaching an approximate 50% growth from 2013. Predictions indicate that by 2020, smartphones and tablets will account for more than 75% of global online commercial transactions, and more than 50% of spend.

The e-commerce market, which is now enhanced by mobile, is comprised primarily of e-commerce companies that have long held a web presence, but have gained even more with the transition from web to mobile.
Continue reading

 

Bank Transfer Extension for Spree

After we started using Spree for development of e-commerce stores for our clients, we often found the clients looking for different modes for achieving more conversion.

From statistics it was visible that end users need more convenience in their hands while shopping online, particularly in the developing nations where credit card penetration is relatively low. In such environment, Bank Transfer appears to be another practical way by which consumers shopping online can make payments for their order.

Vinsol is SpreeCommerce Premier Partner. View Vinsol’s Services

Bank Transfer or BT, is a payment method which gives user the freedom to do an offline/online transaction to the Merchant’s account directly. The user deposits the amount into Merchant’s bank account, which is specified on the checkout page. The end user, on the other hand, after transferring order amount into Merchant’s account, needs to login and provide the payment transaction details with the specific order. Continue reading

 

Custom model callbacks in RubyOnRails

RubyOnRails provides us with many model callbacks around the object’s lifecycle when object is being created, updated or destroyed. For example: before_create, after_create, before_update, after_destroy etc. We use them to write & run our code around the object’s lifecycle by defining a method and associating them as one of the callbacks.

But then how can we make a piece of code execute as a callback for any another defined method except create, update, save and destroy? For example, let’s say we have a model Article and we want to execute something just before and after an article is being published without hooking into model’s before_save and after_save callbacks?

RubyOnRails or more precisely ActiveRecord provides us with a module ActiveModel::Callbacks, which allows us to define and register custom callbacks using define_model_callbacks method. Lets have a look at the snippet below for the above scenario:
Continue reading

 

Reasons to consider Spree when choosing an e-commerce platform

Don’t select your next e-commerce platform without reading about these 13 factors first.

Online business has become a dominant and growing force. Global sales through e-commerce topped $1 trillion in 2012, with North American and Asia-Pacific nations accounting for nearly 70% of it. Projections are that global e-commerce sales will reach $1.5 trillion in 2016.

“Conduct better due diligence of the platform”

Over 60% of medium to large e-commerce site owners claim they should have done better due diligence on the platform they selected for their store. Much too often do businesses face constrained growth or incur significant losses due to misguided platform selections.

ecommerce-development

If you are ready to sell online, choosing the best e-commerce system for your business can be a challenging and confusing process. Based on our deep expertise of rails e-commerce and Spree development, this blog will help make sense of 13 factors you should be considering for selecting an e-commerce platform for your store and how spree stacks up against the competition on those factors.

1. License
There is no “enterprise” edition of Spree – its all the same software. Enterprise clients, small and medium sized businesses can use Open Source Spree for commercial purpose without paying any license fee.
Continue reading

 

Research firm SourcingLine (Clutch) publishes Vinsol in the list of top Ruby on Rails developers across the globe

The new research identifies leading web & application developers that work extensively with Ruby on Rails.

WASHINGTON, Updated At: June 25, 2014 – Today SourcingLine (now Clutch) published new research on leading web and application developers with extensive Ruby on Rails experience.

The top firms inaugurated into the Ruby on Rails Developers Leaders Matrix are as follows:

Vinsol, Quick Left, Idyllic Software, philosophie, Burnside Digital, Perfectial, DevMynd Software, CognitiveClouds, and VeriQual.

Analysts reached out to firms with a demonstrated history of successful projects for both domestic international clients. Official findings are based on a firm’s proven experience, positive client reviews, and market presence. Companion directories were also published, which allow buyers to search for vendors that best fit their project needs and requirements.

“The flexibility and rapid deployment speed of Ruby on Rails make it ideal for many custom development projects,” explained Joshua Margolin, Senior Analyst at SourcingLine. “The vendors in our research have a proven record of using Rails to deliver viable, high-quality solutions to their clients.”
Continue reading

 

Exporting large data to CSV using mysql outfile in multi-host Rails env.

In one of our current project we had a requirement where we needed to allow the site admin to download raw DB data as CSV file for some offline analysis. So, according to the requirement, we allowed the admin to select a database table and a date range to download the generated CSV data.

Our initial approach was to send the CSV file asynchronously using delayed job. It worked like this:
Admin submits the request, system queues it and sends admin an email with the attached CSV file. The reason we did this asynchronously was to avoid timeout during CSV generation.

While testing & benchmarking this approach, we ecountered two issues when we had a large data to export.

  • Too much memory was used by the ruby process when generating the CSV
  • CSV size could be larger than the supported attachment size by the email service provider, hence could result in an undelivered email

Continue reading

 

Generate paperclip missing styles on the fly

Like most developers, who use paperclip gem with their RubyOnRails applications to support picture uploading and generation of distinct styles/thumbnails of pictures, for displaying them in various views, we also do the same in a lot of our apps.

After running one such web app in production for 2 years, the app needed to be redesigned. This required us to generate couple more styles/sizes for each uploaded image.

Paperclip provides a method ‘reprocess!(:style)’ to generate all/one style(s).

Now, when we planned to move this new version of the app to production, we had to generate those missing styles. There were a few thousand thumbnails to be generated, and our estimated time to generate these missing styles was around half an hour. Since we could not have the site available to users without the graphics, we had to put the site in maintenance mode until the new styles got generated. We also realized that as more images get uploaded and used in the system, this downtime if we ever needed to resize graphics again would go up. We could also not do this conversion without taking the site down as more images may get uploaded while we might be resizing using a script.
Continue reading

 

Full-Calendar Rails Engine

A couple of years ago we integrated jQuery FullCalendar plugin with a rails backend. The plugin became reasonably popular and was being used in many Rails apps.

Recently we upgraded it to include an efficient full-sized, drag & drop calendar functionality within a Rails Engine and release it as a gem. With Rails 4 deprecating Rails::Plugin(commit) this would come really handy.

It easily blends within a rails application. Provides a great UI experience with the use of AJAX to fetch events on-the-fly for each month and hooks for user-triggered events (like clicking or dragging an event) for an easy event editing functionality. The view can be easily customized using the configurations provided.

A demo version of this engine implementation can be viewed at http://vinsol.com/fullcalendar-demo

Work in progress : Optimizing Recurring Event functionality.

Source code is hosted on github. Feel free to fork and send a pull request etc.

 

Guide to Caching in Rails using Memcache

Guide to Caching in Rails using Memcache 1

The post intends to cover the topics and tools that can help in implementing Memcache as caching store in Rails and debugging issues with it.

To help newbies grasp it from start, it also provides references to installing and validating the memcache install without Rails too.

“Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering”

Installing Memcache

  • Mac OS X Snow Leopard
    • brew install memcached
  • Ubuntu
    • sudo apt-get install memcached

More information about Installing Memcache can be found here

Starting Memcache

  • memcached -vv

-v option controls verbosity to STDOUT/STDERR. Multiple -v’s increase verbosity. Memcache Settings can be inspected in detail using echo "stats settings" | nc localhost 11211. Get the thirst quenched more at http://linux.die.net/man/1/memcached or man memcached for all options.

Connecting to Memcache Server through Telnet

  • telnet localhost 11211 can be used to connect locally to run/inspect memcache data store
  • Its recommended to follow Using Memcache Commands to get a full hang of memcache commands
  • For the sake of completeness, here are three main commands (store/retrieve/remove key/values)
    • Write to Memcache (Value need to be entered in new line)
      • set abc 0 60 5 # where abc is the key, 60 is duration in seconds and 5 is length of data
      • hello # data to store in abc key
    • Read from Memcache
      • get abc
    • Delete from Memcache
      • delete abc

Implementing and Using in Rails

Installing Dalli (Memcache Client)

Configure Memcache as cache store in Rails

  • Set perform_caching and specify cache_store in the environment file (e.g. RAILS_ROOT/config/environments/production.rb)
    • config.action_controller.perform_caching = true
    • config.cache_store = :dalli_store, { :namespace => “my_project”, :expires_in => 1.day, :socket_timeout => 3, :compress => true }
      • Detailed explanation of Dalli Client options can be found here

Getting Ready for the Fight/Debugging

  • IRB as always helps us get started:
    • require ‘rubygems’
    • require ‘dalli’
    • CACHE = Dalli::Client.new(’127.0.0.1′, { :namespace => “my_project”, :expires_in => 3600, :socket_timeout => 3, :compress => true }) # Options are self-explanatory
    • CACHE.set(‘key’, ‘value’, 180) # last option is expiry time in seconds
    • CACHE.get(‘key’) 2

  • Memcache does not provide any command to list all keys present but at times seeing the keys and their expiry time can be a life saviour. Here is a Ruby Script to list all memcache keys

    Sample Output:

    id expires bytes cache_key
    1 2014-01-22 19:05:09 +0530 5 my_project:key

Using Memcache to store expensive calculations and avoid recalculations

  • We can store expensive calculations in memcache and retrieve it next time if expensive calculation expected to return the same result else recalculate and store again.
  • Rails provides helpers like Rails.cache.read to read cache, Rails.cache.write to write in cache and Rails.cache.fetch to return result if present in cache else write in cache and return the result
  • Code Snippet to pass a block to evaluate if key not present in cache, else return the value of key

      # File: application_controller.rb
    
      # Always Calculate if caching is disabled
      # Calculate the result if key not present and store in Memcache
      # Return calculated result from Memcache if key is present
    
      def data_cache(key, time=2.minutes)
        return yield if caching_disabled?
        output = Rails.cache.fetch(key, {expires_in: time}) do
          yield
        end 
        return output
      rescue
         # Execute the block if any error with Memcache
         
         return yield
      end
    
      def caching_disabled?
        ActionController::Base.perform_caching.blank?
      end
    
  • Using data_cache helper created in last step in application_controller.rb

      # File: posts_controller.rb
    
      respond_to :json
    
      def index
          # Create cache to expire after 3 minutes
          all_posts_for_a_category = data_cache("#{@category.name}", 3.minutes) do
              @category.posts.all
          end
          respond_with(posts: all_posts_for_a_category)
      end
    
      # Post#index - Before Filter
      def find_category
          @category = Category.where(id: params[:category_id]).first
          respond_with({message: 'Category Not Found'}, status: 404) if @category.blank? 
      end
    

Clearing Cache

Auto-Expire after a specified time

  • Memcache accepts an expiration time 3 option with the key to write and only returns the stored result if key has not expired yet, else returns nothing.
    • Rails.cache.write(key, value, {expires_in: time})
    • Rails.cache.fetch(key, {expires_in: time}) { # Block having expensive calculations }

Faking Regex based Expiry

  • Unfortunately, Memcache does not allow listing of all keys so, apparently, there is no way to partially expire a subset of keys. Memcache only provides a delete method for the specified key restricting us to, beforehand, know the key to delete.
  • But, there is a nice trick to partially expire a subset of keys Faking Regex Based Cache keys in Rails 4.
    • To achieve it, we have to focus more on keyword subset and defining subset is the Nirvana to crack the problem.
    • Introduce a dynamic component in all the keys which we want to treat as subset
    • Have a way to control the change of dynamic component
    • When dynamic component changes, a new key is created and stored in Memcache which would be referred for data from now.
    • The old key still remains in memcache and has still not expired but it stores the stale data and fresh data is hold by new key which we are referring in all of our interactions.
    • Memcache would eventually kick the old key with stale data out (uses LRU algorithm to allocate space for new data)
    • In the snippet below, we have added dynamic component last_cache_refreshed_at in Category which can be selectively changed to expire cache of particular categories instead of all

        # File: posts_controller.rb
      
        def index
            # Update Category#last_cache_refreshed_at anytime to force the block evaulation
            # again and storing the new key, hence expiring the last key by effectively not
            # refering it for data and consulting new key for fresh data
            # Time converted to integer to avoid sanitization hassle
      
            all_posts_for_a_category = data_cache("#{@category.name}-#{@category.last_cache_refreshed_at.to_i}", 3.minutes) do
                @category.posts.all
            end
            respond_with(posts: all_posts_for_a_category)
        end
      

Writing Test Cases

  • Enable Caching with Rspec metadata

      # File: spec/spec_helper.rb
    
      # describe "Action", caching: true {}
      # Above blocks would toggle the caching behaviour
      # default caching behavior is OFF
      # Use `bundle exec rspec -t caching:true spec` to run only caching examples
    
      config.around(:each, :caching) do |example|
        caching = ActionController::Base.perform_caching
        ActionController::Base.perform_caching = example.metadata[:caching]
        example.run
        ActionController::Base.perform_caching = caching
      end
    
      # Using `bundle exec rspec spec` would exclude caching:true examples
      # to run from test suite
    
      config.filter_run_excluding :caching => true
    
  • Caching Test Example (Dependency: Memcache should be running)

      # File: spec/controllers/posts_controller_spec.rb
    
      # Use `bundle exec rspec -t caching:true spec` to run only caching examples
    
      context 'Caching Enabled', caching: true do
        before(:each) do
          # last_cache_refreshed_at would ideally be handled through before_create callback
          # by setting current time as value to it
          # Assumption: Current Time in seconds = 1271325600
    
          @category = Category.new(name: 'Ruby', last_cache_refreshed_at: Time.current)
          @category.save.should be_true
          @post = @categroy.posts.build(content: 'Expiring Cache is tough !!!')
          @post.save.should be_true
        end
    
        after(:each) do
          @post.delete
          @category.delete
        end
    
        it "should return posts" do
          @category.last_cache_refreshed_at.should eq(1271325600)
    
          # Check Memcache doesnot have key
          Rails.cache.fetch("Ruby-1271325600").should be_nil
          get :index, {category_id: 1}
          response.should be_ok
          response.body.should == {success: true, posts: @all_posts}.to_json
    
          # As caching is enabled, key should be created in memcache
          Rails.cache.fetch("Ruby-1271325600").should_not be_nil
    
          # Invoke an action to refer a different key in memcache for same action
          # Assumption: Current Time in seconds = 1456782340
          @category.update_attributes(last_cache_refreshed_at: Time.current).should be_true
          @category.last_cache_refreshed_at.should_not eq(1271325600)
    
          # Old key is still present in memcache as still not expired
          Rails.cache.fetch("Ruby-1271325600").should_not be_nil
          get :index, {category_id: 1}
          response.should be_ok
    
          # New key is generated even though old key has still not expired
          # after the specified time, old key would expire
          Rails.cache.fetch("Ruby-1456782340").should_not be_nil
        end
      end
    

Gotchas

  • Using touch command to update expiry cache time (Fixed in Memcache 1.4.14)
  • Check Memcache Version using telnet client
    • Execute stats
    • Look for STAT version 1.4.13

  1. Caching Expensive Api Requests Results

  2. If Object is saved in Memcache, Cache.get(‘key’) might raise

    Dalli::UnmarshalError: Unable to unmarshal value: undefined class/module <model_name>

    Use require_dependency "app/models/.rb"

  3. Expiry Time of 0 makes the key stays for indefinite time only limited by Memcache clearing it out in case of cleaning up

  4. Scoped Expiration AND Reload Namespace using a Proc