Hello World

Be Happy!

ruby multi process with fork



Why I didn't use thread

I use ruby fork function for Multi process
First time I used multiple thread but multiple thread is only run on single core
only single core usage is 100%.
another core is not used by multiple thread.
because MRI has GIT (Global Interpreter Lock)

I did use mutiple process with fork

I change multiple thread to multiple process. that's why I want save time and using all multiple core
And it's well working on multiple core.
my laptop has physical 4 core. multiple process used all core usage 100% and total usage is 400%(4core)

this source for http image download and get image width, height
and print console if image has not specific width, height
require 'mini_magick'
require 'csv'

def image_size_check(datas)
  i = 0
  num = datas.length

  while i < num  do
    data = datas[i]
    begin
      url = data[:thumbnailurl].strip
      image = MiniMagick::Image.open(url)
      width = image[:width]
      height = image[:height]
      puts "#{i},#{width},#{height},#{data[:pageid]},#{data[:menuid]},#{data[:itemid]},#{data[:namekr]},#{data[:thumbnailurl]}" unless (width == 372 and height == 209) or (width == 289 or height == 419)
    rescue StandardError => e
      puts "#{i},#{width},#{height},#{data[:pageid]},#{data[:menuid]},#{data[:itemid]},#{data[:namekr]},#{data[:thumbnailurl]},exception"
    end
    i +=1
  end
end


data = Array.new

CSV.foreach("item_thumbnail.csv", { encoding: "UTF-8", headers: true, header_converters: :symbol, converters: :all}) do |row|
  data << row.to_hash
end

data.each_slice(3000) {|a|
  Process.fork do
    image_size_check(a)
  end
}
Process.wait

Run

$ run image_size_check.rb >> out

Monitoring

$ tail -f out
#thread (1) #process (1) #ruby (13)
List