优化 Rails API 控制器以提高性能

Optimizing Rails API Controller for Improved Performance

提问人:Ridham Patel 提问时间:10/11/2023 最后编辑:Ridham Patel 更新时间:10/29/2023 访问量:68

问:

module Api
  module V1
    class AngularDashboardApisController < BaseController
      include RolesHelper
      before_action :verify_auth_token
      before_action -> { verify_user_permission(2) }

  def dashboard_data_a
    @user = User.find_by(id: params[:user_id])

    unless @user
      return json_response('Bad Request', false, [], :bad_request)
    end

    get_con_tech

    if !params[:from_date].present? && !params[:to_date].present?
      params[:from_date] = DateTime.now.beginning_of_month.to_date
      params[:to_date] = Date.today
    else
      params[:from_date] = params[:from_date].to_date
      params[:to_date] = params[:to_date].to_date
    end

    all_years = Lead.pluck(:created_at).map { |x| x.year }.uniq
    
    check_role(@user, params)
    filter_by_scope
    set_old_and_new_country_tech_hash_A_and_B
    graph_data
    lead_user_summary

    json_success_send_dashboard_graph_response('Dashboard-A', true,@countries_A, @countries_B, @countries_C,@country_total_A, @country_total_B,@country_total_C, @country_tech_array_hash_A, @country_tech_array_hash_B, @country_tech_array_hash_C,Hash[@technology_hash_A.sort_by{|k, v| v}].keys, Hash[@technology_hash_B.sort_by{|k, v| v}].keys, Hash[@technology_hash_C.sort_by{|k, v| v}].keys,@graphdata_lst, @Summary,all_years, :ok)
  end
  
  def graph_data
    graph_lead = sort_by_date(@country_tech_graph)
    @graphdata_map = Hash.new(0) # Initialize a default value of 0 for missing keys
    @graphdata_lst = []
  
    if params[:mode] == 'month' || params[:mode] == 'day' || params[:mode] == 'week'
      graph_lead.each do |lead|
        date_key = case params[:mode]
                   when 'month'
                     month_map[lead.date.month.to_s]
                   when 'day', 'week'
                     lead.date.to_date
                   end
  
        @graphdata_map[date_key] += 1
      end
    end
  
    @graphdata_map.each do |key, value|
      @graphdata_lst.push({ name: "#{key}", value: "#{value}", tooltip: "#{key}" })
    end
  end      

  def filter_by_scope
    @country_tech_hash_A, @country_tech_hash_A_graph, @country_tech_hash_id_A = filter_and_group_leads(@technology_hash_A, params)
    @country_tech_hash_B, @country_tech_hash_B_graph, @country_tech_hash_id_B = filter_and_group_leads(@technology_hash_B, params)
    @country_tech_hash_C, @country_tech_hash_C_graph, @country_tech_hash_id_C = filter_and_group_leads(@technology_hash_C, params)

    @country_tech_graph = @country_tech_hash_A + @country_tech_hash_B + @country_tech_hash_C

    @country_tech_hash_A = @country_tech_hash_A_graph
    @country_tech_hash_B = @country_tech_hash_B_graph
    @country_tech_hash_C = @country_tech_hash_C_graph
  end

  def set_old_and_new_country_tech_hash_A_and_B
    @new_country_tech_hash_id_A = {}
    @country_tech_hash_id_A.each do |key,value|
      if key[0] == nil
        key[0] = 'unknown'
      end
      k=key[0]+"", ""+key[1]
      @new_country_tech_hash_id_A[key[2]]=k
    end
    @new_country_tech_hash_id_A = @new_country_tech_hash_id_A.each_with_object({}) { |(k,v),g| (g[v] ||= []) << k }

    @country_tech_array_hash_A = {}
    @country_tech_array_A = []
    @countries_A = Set.new
    @countries_B = Set.new
    @countries_C = Set.new
    index = 0
    @klass = Klass.find_by(name: "Lead")
    @fields = @user.fields_for_table_with_order1(klass: @klass)
    @country_tech_hash_A.each do |key,value|
      array = []
      @countries_A.add(key[0])
      ids = @new_country_tech_hash_id_A[[key[0],key[1]]]
      @leads = Lead.where('id in (?)', ids).select(@fields.pluck(:name))
      if @country_tech_array_hash_A.key? (key[1])
        @country_tech_array_hash_A[key[1]] = @country_tech_array_hash_A[key[1]].push(array.push(key[0],value,@leads))
      else
        @country_tech_array_A = []
        @country_tech_array_hash_A[key[1]] = @country_tech_array_A.push(array.push(key[0],value,@leads))
      end
      @country_hash_A[key[0]] = index
      index = index + 1
      if @country_total_A.key? (key[0])
        @country_total_A[key[0]] = @country_total_A[key[0]] + value
      else
        @country_total_A[key[0]] =  value
      end
    end

    @new_country_tech_hash_id_B = {}
    @country_tech_hash_id_B.each do |key,value|
      if key[0] == nil
        key[0] = 'unknown'
      end
      k=key[0]+"", ""+key[1]
      @new_country_tech_hash_id_B[key[2]]=k
    end
    @new_country_tech_hash_id_B = @new_country_tech_hash_id_B.each_with_object({}) { |(k,v),g| (g[v] ||= []) << k }
    @country_tech_array_hash_B = {}
    @country_tech_array_B = []
    @country_tech_hash_B.each do |key,value|
      array = []
      @countries_B.add(key[0])
      ids = @new_country_tech_hash_id_B[[key[0],key[1]]]
      @leads = Lead.where('id in (?)', ids).select(@fields.pluck(:name))
      if @country_tech_array_hash_B.key? (key[1])
        @country_tech_array_hash_B[key[1]] = @country_tech_array_hash_B[key[1]].push(array.push(key[0],value,@leads))
      else
        @country_tech_array_B = []
        @country_tech_array_hash_B[key[1]] = @country_tech_array_B.push(array.push(key[0],value,@leads))
      end
      @country_hash_B[key[0]] = index
      index = index + 1
      if @country_total_B.key? (key[0])
        @country_total_B[key[0]] = @country_total_B[key[0]] + value
      else
        @country_total_B[key[0]] =  value
      end
    end

    @new_country_tech_hash_id_C = {}
    @country_tech_hash_id_C.each do |key,value|
      if key[0] == nil
        key[0] = 'unknown'
      end
      k=key[0]+"", ""+key[1]
      @new_country_tech_hash_id_C[key[2]]=k
    end
    @new_country_tech_hash_id_C = @new_country_tech_hash_id_C.each_with_object({}) { |(k,v),g| (g[v] ||= []) << k }
    @country_tech_array_hash_C = {}
    @country_tech_array_C = []
    @country_tech_hash_C.each do |key,value|
      array = []
      @countries_C.add(key[0])
      ids = @new_country_tech_hash_id_C[[key[0],key[1]]]
      @leads = Lead.where('id in (?)', ids).select(@fields.pluck(:name))
      if @country_tech_array_hash_C.key? (key[1])
        @country_tech_array_hash_C[key[1]] = @country_tech_array_hash_C[key[1]].push(array.push(key[0],value,@leads))
      else
        @country_tech_array_C = []
        @country_tech_array_hash_C[key[1]] = @country_tech_array_C.push(array.push(key[0],value,@leads))
      end
      @country_hash_C[key[0]] = index
      index = index + 1
      if @country_total_C.key? (key[0])
        @country_total_C[key[0]] = @country_total_C[key[0]] + value
      else
        @country_total_C[key[0]] =  value
      end
    end
  end

  def get_con_tech
    @country_hash_A = {}
    @country_hash_B = {}
    @country_hash_C = {}
    @technology_hash_A = {}
    @technology_hash_B = {}
    @technology_hash_C = {}

    @technology_A = CountryTechnology.where(group:'A',country_technology:'technology')
    @technology_B = CountryTechnology.where(group:'B',country_technology:'technology')
    @technology_C = CountryTechnology.where(group:'C',country_technology:'technology')

    @technology_A.each do |technology|
      @technology_hash_A[technology.name] = technology.position
    end

    @technology_B.each do |technology|
      @technology_hash_B[technology.name] = technology.position
    end

    @technology_C.each do |technology|
      @technology_hash_C[technology.name] = technology.position
    end
      @country_total_A = Hash[@country_hash_A.sort_by{|k, v| v}]
      @country_total_B = Hash[@country_hash_B.sort_by{|k, v| v}]
      @country_total_C = Hash[@country_hash_C.sort_by{|k, v| v}]
      @country_total_A.each { |k, v| @country_total_A[k] = 0 }
      @country_total_B.each { |k, v| @country_total_B[k] = 0 }
      @country_total_C.each { |k, v| @country_total_C[k] = 0 }
  end

  def lead_user_summary
    @LeadgroupByUser = Lead.filter_by_from_to_date_tech(params[:from_date], params[:to_date], @technology_hash_A.keys + @technology_hash_B.keys + @technology_hash_C.keys)
  
    filters = {
      source: params[:source],
      is_closed: params[:is_closed],
      live_chat_handle_name: params[:live_chat_handle_name],
      if_paid: params[:if_paid],
      if_organic: params[:if_organic],
      supportdesk: params[:supportdesk],
      terminate_reasons: params[:terminate_reasons],
      device: params[:device],
      user_id: params[:assign_to],
      technology: params[:technology]
    }
  
    filters.each do |filter_name, filter_value|
      @LeadgroupByUser = @LeadgroupByUser.send("filter_by_#{filter_name}", filter_value) if filter_value.present? && filter_value != "All"
    end
  
    @Summaryhash = @LeadgroupByUser.group(:user_id).count
    @LeadgroupByUser1 = @LeadgroupByUser.filter_by_is_closed("Yes")
    @Summaryhash1 = @LeadgroupByUser1.group(:user_id).count
    @Summary = []
    @klass = Klass.find_by(name: "Lead")
    @fields = @user.fields_for_table_with_order1(klass: @klass)
  
    @Summaryhash.each do |key, value|
      total_closed_count = @Summaryhash1[key].present? ? @Summaryhash1[key] : 0
      @assignedLeads1 = @LeadgroupByUser.where(user_id: key).select(:id, @fields.pluck(:name))
      @Summary.push({'key': key, 'total_assigned_count': value, 'total_closed_count': total_closed_count, 'assigned_leads': @assignedLeads1, 'labels': @fields.pluck(:name)})
    end
  end

  private

  def sort_by_date(arr)
    arr.sort_by { |h| h["date"].to_date.to_s.split('-') }
  end

  def filter_and_group_leads(technology_hash, params)
    filtered_leads = Lead.filter_by_from_to_date_tech(params[:from_date], params[:to_date], technology_hash.keys)
    filtered_leads = filtered_leads.filter_by_source(params[:source]) if params[:source].present? && params[:source] != "All"
    filtered_leads = filtered_leads.filter_by_is_closed(params[:is_closed]) if params[:is_closed].present? && params[:is_closed] != "All"
    filtered_leads = filtered_leads.filter_by_livechathandle(params[:live_chat_handle_name]) if params[:live_chat_handle_name].present? && params[:live_chat_handle_name] != "All"
    filtered_leads = filtered_leads.filter_by_if_paid(params[:if_paid]) if params[:if_paid].present? && params[:if_paid] != "All"
    filtered_leads = filtered_leads.filter_by_if_organic(params[:if_organic]) if params[:if_organic].present? && params[:if_organic] != "All"
    filtered_leads = filtered_leads.filter_by_supportdesk(params[:supportdesk]) if params[:supportdesk].present? && params[:supportdesk] != "All"
    filtered_leads = filtered_leads.filter_by_terminate_reasons(params[:terminate_reasons]) if params[:terminate_reasons].present? && params[:terminate_reasons] != "All"
    filtered_leads = filtered_leads.filter_by_device(params[:device]) if params[:device].present? && params[:device] != "All"
    filtered_leads = filtered_leads.filter_by_user_id(params[:assign_to]) if params[:assign_to].present? && params[:assign_to] != "All"
    filtered_leads = filtered_leads.filter_by_technology(params[:technology]) if params[:technology].present? && params[:technology] != "All"
  
    graph_data = filtered_leads.group(:country, :technology).count
    country_tech_hash_id = filtered_leads.group(:country, :technology, :id).count

    return filtered_leads, graph_data, country_tech_hash_id
  end
end
end
end

我有这个从数据库中检索数据的 Rails API 控制器,但我遇到了性能问题。API 每次获取相同的数据需要不同的时间。我正在寻找有关优化代码以获得更好性能的建议。

我试图通过重构代码来优化dashboard_data_a方法以获得更好的性能。具体来说,我重构了数据库查询,并为一些经常访问的数据添加了缓存。

此外,我还创建了用于过滤的通用函数,即 filter_and_group_leads(),但这不会降低我的时间复杂性,因此我预计这些优化将显着缩短响应时间,特别是对于检索 Dashboard-A 数据的请求。然而,虽然略有改善,但响应时间仍然有很大差异,我相信还有进一步改进的空间。

Ruby-on-Rails Ruby 数据库 控制器 数据过滤

评论

1赞 spickermann 10/11/2023
如果没有有关数据库架构的外观、表的大小、是否有索引等的更多信息,我们不太可能为您提供帮助。作为第一步,我建议将 APM 工具(如 NewRelic、Sentry、Skylight 或 AppSignal)添加到您的应用程序中,以收集更多数据: 是否存在导致响应缓慢的特定查询参数模式?或者哪个步骤实际上是最慢的:一个特定的数据库查询还是太多的数据库查询?N+1 问题?数据序列化慢?
1赞 max 10/11/2023
我真的会从重构这段代码开始,并将复杂的操作移出控制器。对不起,我的直言不讳,但这段代码的质量非常糟糕。如果将复杂的过滤操作移出控制器,则对每个单独的组件进行基准测试并查找问题会容易得多。blog.appsignal.com/2021/04/14/......
0赞 tadman 10/11/2023
这里有相当多的代码需要审查,这远远超出了 Stack Overflow 问题的范围。分析事物。查找热点。探讨。如果你有一个特定的操作被证明是顽固的,那么把问题集中在这个问题上。
0赞 tadman 10/11/2023
注意:Ruby 是一种区分大小写的语言,大写字母在语法方面具有特定的含义。变量和方法名称应仅为小写字母。大写字母表示形式的常量或 。此约定也适用于类型实例变量,如 in 通常写为 .ClassNameCONSTANT_NAME@@LeadgroupByUser@leadgroup_by_user
0赞 tadman 10/11/2023
这里突出的一件事是名称以 、 或 、 等结尾的变量,这意味着您需要的是数组哈希值,具体取决于用例。将数据组织成结构,不要使用一堆变量。12AB

答:

0赞 Julian 10/26/2023 #1

你的编码风格太冗长了。

你听说过桑迪梅斯吗?她正在阐述编码风格。她说,例如,一种方法最多必须有 5 行。因此,将方法分解为更小的可理解块在某种程度上是游戏。

您的方法有 98 行,这太疯狂了,无法查看和识别您遇到的任何问题,尤其是性能问题。我会再次处理这个问题并编辑您的问题。我的控制器的总长度只有它的一半,包括白线。因此,制作更小的方法也将帮助您识别性能问题。set_old_and_new_country_tech_hash_A_and_B

在当前状态下,帮助您提高代码库的性能在某种程度上是不可能的。事实上,这是我一生中见过的最长的控制器动作,我在 Ruby On Rails 上工作了 15 年!

0赞 Eric Prostko 10/26/2023 #2

问题在于代码生成的数据库调用数量之多。

但更一般地说,您将使用基准测试工具开始该过程,该工具可以测量运行代码所需的实际时间。我倾向于使用 2 种工具:BenchmarkRackMiniProfiler

然后,一旦你有了代码当前状态的基线,你就会尝试进行增量更改,并确保这些更改实际上在基准测试中花费的时间更少。

现在尝试为您做出一些具体的更改和建议。

  1. 您是否检查过要查找的每个字段都由数据库索引支持?
  2. 我注意到您没有使用任何急切的数据加载,除非我遗漏了某些内容。您是否尝试过通过预先加载解决任何 n+1 问题?