Opened 10 years ago

Closed 10 years ago

#92 closed defect (fixed)

Labwiki times out if I have too many slices

Reported by: johren@bbn.com Owned by: johren@bbn.com
Priority: major Milestone: GEC21
Component: Labwiki Version: Sprint2
Keywords: Cc: divyashri.bhat@gmail.com, thierry.rakotoarivelo@nicta.com.au
Dependencies:

Description

A couple of the users that I used for testing (johren and gemini70) are members of a project (joIMAutoTest) that I use for scale testing both IM projects. This project currently has 55 slices (most are empty). When I try to login to Labwiki with one of these users, I get the timeout error below. It looks by the stack trace that it is filling a buffer when it tries to load the projects and slices. This could be a problem at GEC if they are a member of the GEC20 project and it has many slices.

Timeout::Error at / Timeout::Error Ruby /home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/protocol.rb: in rescue in rbuf_fill, line 146 Web GET gimi1.casa.umass.edu/ Jump to:

GET POST Cookies ENV

Traceback (innermost first)

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/protocol.rb: in rescue in rbuf_fill

raise Timeout::Error...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/protocol.rb: in rbuf_fill

begin...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/protocol.rb: in readuntil

rbuf_fill...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/protocol.rb: in readline

readuntil("\n").chop...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in read_status_line

str = sock.readline...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in read_new

httpv, code, msg = read_status_line(sock)...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in block in transport_request

res = HTTPResponse.read_new(@socket)...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in catch

res = catch(:response) {...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in transport_request

res = catch(:response) {...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in request

res = transport_request(req, &block)...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in block in request

return request(req, body, &block)...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in start

return yield(self)...

/home/gimiadmin/.rvm/rubies/ruby-1.9.3-p545/lib/ruby/1.9.1/net/http.rb: in request

start {...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/httparty-0.11.0/lib/httparty/request.rb: in perform

self.last_response = http.request(@raw_request) do |http_response|...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/httparty-0.11.0/lib/httparty.rb: in perform_request

Request.new(http_method, path, options).perform(&block)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/httparty-0.11.0/lib/httparty.rb: in get

perform_request Net::HTTP::Get, path, options, &block...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/httparty-0.11.0/lib/httparty.rb: in get

Basement.get(*args, &block)...

/home/gimiadmin/labwiki/lib/labwiki/session_init.rb: in find_or_create

obj = HTTParty.get("#{ges_url}/#{res_path}/#{res_id}")...

/home/gimiadmin/labwiki/lib/labwiki/session_init.rb: in block in update_geni_projects_slices

proj = find_or_create("projects", p[:name], { irods_user: OMF::Web::SessionStore?[:id, :irods_user] })...

/home/gimiadmin/labwiki/lib/labwiki/session_init.rb: in each

OMF::Web::SessionStore?[:projects, :geni_portal].each do |p|...

/home/gimiadmin/labwiki/lib/labwiki/session_init.rb: in update_geni_projects_slices

OMF::Web::SessionStore?[:projects, :geni_portal].each do |p|...

/home/gimiadmin/labwiki/lib/labwiki/session_init.rb: in call

update_geni_projects_slices(user) if LabWiki::Configurator[:gimi]...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb: in block in call

@app.call(env)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb: in catch

result = catch(:warden) do...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/warden-1.2.3/lib/warden/manager.rb: in call

result = catch(:warden) do...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/rack-openid-1.4.2/lib/rack/openid.rb: in call

status, headers, body = @app.call(env)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/rack-1.5.2/lib/rack/session/abstract/id.rb: in context

status, headers, body = app.call(env)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/rack-1.5.2/lib/rack/session/abstract/id.rb: in call

context(env)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/rack-1.5.2/lib/rack/showexceptions.rb: in call

@app.call(env)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/rack-cors-0.2.9/lib/rack/cors.rb: in call

status, headers, body = @app.call env...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/connection.rb: in block in pre_process

response = @app.call(@request.env)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/connection.rb: in catch

catch(:async) do...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/connection.rb: in pre_process

catch(:async) do...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/websocket-rack-0.4.0/lib/rack/websocket/extensions/thin/connection.rb: in pre_process_with_websocket

pre_process_without_websocket...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/connection.rb: in process

post_process(pre_process)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/connection.rb: in receive_data

process if @request.parse(data)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/websocket-rack-0.4.0/lib/rack/websocket/extensions/common.rb: in receive_data_with_websocket

receive_data_without_websocket(data)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/websocket-rack-0.4.0/lib/rack/websocket/extensions/common.rb: in receive_data_with_flash_policy_file

receive_data_without_flash_policy_file(data)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/eventmachine-1.0.3/lib/eventmachine.rb: in run_machine

run_machine...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/eventmachine-1.0.3/lib/eventmachine.rb: in run

run_machine...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/backends/base.rb: in start

EventMachine?.run(&starter)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/server.rb: in start

@backend.start { setup_signals if @setup_signals }...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/controllers/controller.rb: in start

server.start...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/runner.rb: in run_command

controller.send(@command, *@arguments)...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/gems/thin-1.6.2/lib/thin/runner.rb: in run!

run_command...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/bundler/gems/omf_web-fc3b04fb44c5/lib/omf-web/thin/runner.rb: in run!

super...

/home/gimiadmin/labwiki/vendor/ruby/1.9.1/bundler/gems/omf_web-fc3b04fb44c5/lib/omf_web.rb: in start

runner.run!...

../lib/labwiki/start.rb: in <main>

OMF::Web.start(opts)...

Request information GET

No GET data. POST

No POST data. COOKIES Variable Value labwiki.session.4000 "BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw\nZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki\nCHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy\nLmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl\ncnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg==\n--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe" Rack ENV Variable Value GATEWAY_INTERFACE CGI/1.2 HTTP_ACCEPT text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 HTTP_ACCEPT_ENCODING gzip, deflate HTTP_ACCEPT_LANGUAGE en-US,en;q=0.5 HTTP_CONNECTION keep-alive HTTP_COOKIE labwiki.session.4000=BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw%0AZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki%0ACHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy%0ALmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl%0AcnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg%3D%3D%0A--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe HTTP_HOST gimi1.casa.umass.edu:4000 HTTP_ORIGIN nil HTTP_USER_AGENT Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:29.0) Gecko/20100101 Firefox/29.0 HTTP_VERSION HTTP/1.1 PATH_INFO / QUERY_STRING REMOTE_ADDR 128.33.251.134 REQUEST_METHOD GET REQUEST_PATH / REQUEST_URI / SCRIPT_NAME SERVER_NAME gimi1.casa.umass.edu SERVER_PORT 4000 SERVER_PROTOCOL HTTP/1.1 SERVER_SOFTWARE thin 1.6.2 codename Doc Brown async.callback #<Method: Thin::Connection#post_process> async.close #<EventMachine::DefaultDeferrable:0x00000004a86b60> async.connection #<Thin::Connection:0x00000005998298 @signature=151003, @request=#<Thin::Request:0x00000004a885f0 @parser=#<Thin::HttpParser:0x00000004a885a0>, @data=nil, @nparsed=697, @body=#<StringIO:0x00000004a88500>, @env={"SERVER_SOFTWARE"=>"thin 1.6.2 codename Doc Brown", "SERVER_NAME"=>"gimi1.casa.umass.edu", "rack.input"=>#<StringIO:0x00000004a88500>, "rack.version"=>[1, 0], "rack.errors"=>#<IO:<STDERR>>, "rack.multithread"=>false, "rack.multiprocess"=>false, "rack.run_once"=>false, "REQUEST_METHOD"=>"GET", "REQUEST_PATH"=>"/", "PATH_INFO"=>"/", "REQUEST_URI"=>"/", "HTTP_VERSION"=>"HTTP/1.1", "HTTP_HOST"=>"gimi1.casa.umass.edu:4000", "HTTP_USER_AGENT"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:29.0) Gecko/20100101 Firefox/29.0", "HTTP_ACCEPT"=>"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "HTTP_ACCEPT_LANGUAGE"=>"en-US,en;q=0.5", "HTTP_ACCEPT_ENCODING"=>"gzip, deflate", "HTTP_COOKIE"=>"labwiki.session.4000=BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw%0AZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki%0ACHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy%0ALmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl%0AcnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg%3D%3D%0A--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe", "HTTP_CONNECTION"=>"keep-alive", "GATEWAY_INTERFACE"=>"CGI/1.2", "SERVER_PORT"=>"4000", "QUERY_STRING"=>"", "SERVER_PROTOCOL"=>"HTTP/1.1", "rack.url_scheme"=>"http", "SCRIPT_NAME"=>"", "async.connection"=>#<Thin::Connection:0x00000005998298 ...>, "REMOTE_ADDR"=>"128.33.251.134", "async.callback"=>#<Method: Thin::Connection#post_process>, "async.close"=>#<EventMachine::DefaultDeferrable:0x00000004a86b60>, "HTTP_ORIGIN"=>nil, "rack.session"=>{"session_id"=>"afcd502048bec987ed210daf98868f93c20294ca148bac5ee59c9e741afc334d", "sid"=>"s1756428_3341027", "warden.user.default.key"=>"https://portal.geni.net/server/server.php/idpage?user=gemini70"}, "rack.session.options"=>{:path=>"/", :domain=>nil, :expire_after=>nil, :secure=>false, :httponly=>true, :defer=>false, :renew=>false, :sidbits=>128, :secure_random=>SecureRandom?, :secret=>"4000", :coder=>#<Rack::Session::Cookie::Base64::Marshal:0x000000029fea00>}, "rack.request.query_string"=>"", "rack.request.query_hash"=>{}, "warden"=>Warden::Proxy:39071580 @config={:default_scope=>:default, :scope_defaults=>{}, :default_strategies=>{:_all=>[:openid]}, :intercept_401=>true, :failure_app=>AuthFailureApp?}, "rack.request.cookie_hash"=>{"labwiki.session.4000"=>"BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw\nZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki\nCHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy\nLmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl\ncnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg==\n--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe"}, "rack.request.cookie_string"=>"labwiki.session.4000=BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw%0AZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki%0ACHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy%0ALmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl%0AcnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg%3D%3D%0A--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe", "rack.session.unpacked_cookie_data"=>{"session_id"=>"afcd502048bec987ed210daf98868f93c20294ca148bac5ee59c9e741afc334d", "sid"=>"s1756428_3341027", "warden.user.default.key"=>"https://portal.geni.net/server/server.php/idpage?user=gemini70"}}>, @response=#<Thin::Response:0x00000004a884b0 @headers=, @status=200, @persistent=false, @skip_body=false>, @backend=0.0.0.0:4000, @app=#<Rack::Cors:0x0000000272a608 @app=#<Rack::ShowExceptions:0x000000029fe960 @app=#<Rack::Session::Cookie:0x000000029feb40 @secrets=["4000"], @coder=#<Rack::Session::Cookie::Base64::Marshal:0x000000029fea00>, @app=#<Rack::OpenID:0x000000029feb68 @app=#<Warden::Manager:0x000000029fed70 @config={:default_scope=>:default, :scope_defaults=>{}, :default_strategies=>{:_all=>[:openid]}, :intercept_401=>true, :failure_app=>AuthFailureApp?}, @app=#<SessionInit:0x000000029fee10 @app=#<Rack::URLMap:0x000000029f7250 @mapping=[[nil, "/plugin/experiment/create_iwidget", /^\/+plugin\/+experiment\/+create_iwidget(.*)/n, #<Rack::Builder:0x0000000284d288 @run=#<Proc:0x0000000284d198@/home/gimiadmin/labwiki/lib/labwiki/config.ru:190 (lambda)>, @map=nil, @use=[]>], [nil, "/plugin/source_edit/create_script", /\/+plugin\/+source_edit\/+create_script(.*)/n, #<Rack::Builder:0x0000000284d738 @run=#<Proc:0x0000000284d580@/home/gimiadmin/labwiki/lib/labwiki/config.ru:186 (lambda)>, @map=nil, @use=[]>], [nil, "/disconnect_all_db_connections", /\/+disconnect_all_db_connections(.*)/n, #<Rack::Builder:0x0000000284d530 @run=#<Proc:0x0000000284d378@/home/gimiadmin/labwiki/lib/labwiki/config.ru:182 (lambda)>, @map=nil, @use=[]>], [nil, "/resource/plugin", /\/+resource\/+plugin(.*)/n, #<Rack::Builder:0x00000002d80020 @run=#<LabWiki::PluginResourceHandler:0x00000002d870a0 @root=nil, @headers=nil, @default_mime="text/plain">, @map=nil, @use=[]>], [nil, "/resource/vendor", /\/+resource\/+vendor(.*)/n, #<Rack::Builder:0x00000001fdf4e8 @run=#<OMF::Web::Rack::MultiFile:0x00000002d72c68 @root=nil, @headers=nil, @default_mime="text/plain", @roots=["/home/gimiadmin/labwiki/lib/labwiki/../../htdocs", "./resources", "/home/gimiadmin/labwiki/vendor/ruby/1.9.1/bundler/gems/omf_web-fc3b04fb44c5/lib/omf-web/thin/../../../share/htdocs"], @sub_path=["vendor"], @version=true, @version_map={"backbone"=>"backbone-1.0.0", "bootstrap"=>"bootstrap-3.1.1", "codemirror"=>"CodeMirror?-2.32", "d3"=>"d3-3.0", "jquery"=>"jquery-1.7.2", "jquery-ui"=>"jquery-ui-1.8.23", "mustache"=>"mustache-0.7.0", "raphael"=>"raphael-2.1.0", "silk_icons"=>"silk_icons_v013", "slickgrid"=>"slickgrid-200512", "stacktrace"=>"stacktrace-0.3", "underscore"=>"underscore-1.4.4", "require"=>"require-2.1.8", "require-css"=>"require-css-0.0.7", "font-awesome"=>"font-awesome-4.0.3", "smartmenus"=>"smartmenus-0.9.6"}>, @map=nil, @use=[]>], [nil, "/geni_login", /\/+geni_login(.*)/n, #<Rack::Builder:0x00000001fe0be0 @run=#<Proc:0x00000001fe08c0@/home/gimiadmin/labwiki/lib/labwiki/config.ru:89>, @map=nil, @use=[]>], [nil, "/resource", /\/+resource(.*)/n, #<Rack::Builder:0x00000002d87000 @run=#<OMF::Web::Rack::MultiFile:0x00000002d86038 @root=nil, @headers=nil, @default_mime="text/plain", @roots=["/home/gimiadmin/labwiki/lib/labwiki/../../htdocs", "./resources", "/home/gimiadmin/labwiki/vendor/ruby/1.9.1/bundler/gems/omf_web-fc3b04fb44c5/lib/omf-web/thin/../../../share/htdocs"], @version=nil>, @map=nil, @use=[]>], [nil, "/_content", /\/+_content(.*)/n, #<Rack::Builder:0x00000001f02750 @run=#<OMF::Web::Rack::ContentHandler:0x00000002309790 @logger=#<Log4r::Logger:0x00000002310c98 @fullname="development::OMF::Web::Rack::ContentHandler", @outputters=[], @additive=true, @name="ContentHandler", @path="development::OMF::Web::Rack", @parent=#<Log4r::Logger:0x00000000ee2e10 @fullname="development", @outputters=[#<Log4r::StdoutOutputter:0x00000001376fa8 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000001376710>, @name="console", @level=0, @formatter=#<Log4r::PatternFormatter:0x000000013398d8 @depth=7, @pattern="%l %c: %m", @date_pattern="%Y-%m-%d %H:%M:%S", @date_method=nil>, @out=#<IO:<STDOUT>>>], @additive=true, @name="development", @path="", @parent=#<Log4r::RootLogger:0x00000000ee2bb8 @level=0, @outputters=[]>, @level=0, @trace=false>, @level=0, @trace=false>>, @map=nil, @use=[]>], [nil, "/_update", /\/+_update(.*)/n, #<Rack::Builder:0x0000000254f590 @run=#<OMF::Web::Rack::UpdateHandler:0x00000001f028e0>, @map=nil, @use=[]>], [nil, "/labwiki", /\/+labwiki(.*)/n, #<Rack::Builder:0x00000001fe23a0 @run=#<Proc:0x00000001fe15e0@/home/gimiadmin/labwiki/lib/labwiki/config.ru:65>, @map=nil, @use=[]>], [nil, "/_search", /\/+_search(.*)/n, #<Rack::Builder:0x00000002708be8 @run=#<LabWiki::SearchHandler:0x0000000270eea8 @opts={}, @logger=#<Log4r::Logger:0x00000002ebfd00 @fullname="development::LabWiki::SearchHandler", @outputters=[], @additive=true, @name="SearchHandler", @path="development::LabWiki", @parent=#<Log4r::Logger:0x00000000ee2e10 @fullname="development", @outputters=[#<Log4r::StdoutOutputter:0x00000001376fa8 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000001376710>, @name="console", @level=0, @formatter=#<Log4r::PatternFormatter:0x000000013398d8 @depth=7, @pattern="%l %c: %m", @date_pattern="%Y-%m-%d %H:%M:%S", @date_method=nil>, @out=#<IO:<STDOUT>>>], @additive=true, @name="development", @path="", @parent=#<Log4r::RootLogger:0x00000000ee2bb8 @level=0, @outputters=[]>, @level=0, @trace=false>, @level=0, @trace=false>>, @map=nil, @use=[]>], [nil, "/_column", /\/+_column(.*)/n, #<Rack::Builder:0x0000000270ee08 @run=#<LabWiki::ColumnHandler:0x0000000284dbe8 @opts={}>, @map=nil, @use=[]>], [nil, "/logout", /\/+logout(.*)/n, #<Rack::Builder:0x00000001fe06b8 @run=#<Proc:0x00000001fdfa60@/home/gimiadmin/labwiki/lib/labwiki/config.ru:100>, @map=nil, @use=[]>], [nil, "/login", /\/+login(.*)/n, #<Rack::Builder:0x00000001fe14c8 @run=#<Proc:0x00000001fe0cf8@/home/gimiadmin/labwiki/lib/labwiki/config.ru:77>, @map=nil, @use=[]>], [nil, "/_ws", /\/+_ws(.*)/n, #<Rack::Builder:0x00000002d85f98 @run=#<OMF::Web::Rack::WebsocketHandler:0x00000002550d78 @options={}>, @map=nil, @use=[]>], [nil, "", /(.*)/n, #<Rack::Builder:0x0000000284da80 @run=#<Proc:0x0000000284d828@/home/gimiadmin/labwiki/lib/labwiki/config.ru:159>, @map=nil, @use=[]>]]>, @logger=#<Log4r::Logger:0x00000002e55180 @fullname="development::SessionInit", @outputters=[], @additive=true, @name="SessionInit", @path="development", @parent=#<Log4r::Logger:0x00000000ee2e10 @fullname="development", @outputters=[#<Log4r::StdoutOutputter:0x00000001376fa8 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x00000001376710>, @name="console", @level=0, @formatter=#<Log4r::PatternFormatter:0x000000013398d8 @depth=7, @pattern="%l %c: %m", @date_pattern="%Y-%m-%d %H:%M:%S", @date_method=nil>, @out=#<IO:<STDOUT>>>], @additive=true, @name="development", @path="", @parent=#<Log4r::RootLogger:0x00000000ee2bb8 @level=0, @outputters=[]>, @level=0, @trace=false>, @level=0, @trace=false>>>, @store=#<OpenID::Store::Filesystem:0x00000001e83a18 @nonce_dir="/tmp/openid_4000/nonces", @association_dir="/tmp/openid_4000/associations", @temp_dir="/tmp/openid_4000/temp">>, @default_options={:path=>"/", :domain=>nil, :expire_after=>nil, :secure=>false, :httponly=>true, :defer=>false, :renew=>false, :sidbits=>128, :secure_random=>SecureRandom?, :secret=>"4000", :coder=>#<Rack::Session::Cookie::Base64::Marshal:0x000000029fea00>}, @key="labwiki.session.4000", @cookie_only=true, @sidbits=128, @sid_secure=SecureRandom?, @sid_length=32>, @template=#<ERB:0x000000029fe938 @safe_level=nil, @src="#coding:US-ASCII\n_erbout = ''; _erbout.concat \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 4.01 Transitional//EN\\\" \\\"http://www.w3.org/TR/html4/loose.dtd\\\">
n<html lang=
\"en
\">
n<head>
n <meta http-equiv=
\"content-type
\" content=
\"text/html; charset=utf-8
\" />
n <meta name=
\"robots
\" content=
\"NONE,NOARCHIVE
\" />
n <title>\"\n\n\n\n\n; _erbout.concat((h exception.class ).to_s); _erbout.concat \" at \"; _erbout.concat((h path ).to_s); _erbout.concat \"</title>
n <style type=
\"text/css
\">
n html * { padding:0; margin:0; }
n body * { padding:10px 20px; }
n body * * { padding:0; }
n body { font:small sans-serif; }
n body>div { border-bottom:1px solid #ddd; }
n h1 { font-weight:normal; }
n h2 { margin-bottom:.8em; }
n h2 span { font-size:80%; color:#666; font-weight:normal; }
n h3 { margin:1em 0 .5em 0; }
n h4 { margin:0 0 .5em 0; font-weight: normal; }
n table {
n border:1px solid #ccc; border-collapse: collapse; background:white; }
n tbody td, tbody th { vertical-align:top; padding:2px 3px; }
n thead th {
n padding:1px 6px 1px 3px; background:#fefefe; text-align:left;\\n font-weight:normal; font-size:11px; border:1px solid #ddd; }
n tbody th { text-align:right; color:#666; padding-right:.5em; }
n table.vars { margin:5px 0 2px 40px; }
n table.vars td, table.req td { font-family:monospace; }
n table td.code { width:100%;}\\n table td.code div { overflow:hidden; }
n table.source th { color:#666; }
n table.source td {
n font-family:monospace; white-space:pre; border-bottom:1px solid #eee; }
n ul.traceback { list-style-type:none; }
n ul.traceback li.frame { margin-bottom:1em; }
n div.context { margin: 10px 0; }
n div.context ol {
n padding-left:30px; margin:0 10px; list-style-position: inside; }
n div.context ol li {
n font-family:monospace; white-space:pre; color:#666; cursor:pointer; }
n div.context ol.context-line li { color:black; background-color:#ccc; }
n div.context ol.context-line li span { float: right; }
n div.commands { margin-left: 40px; }
n div.commands a { color:black; text-decoration:none; }
n #summary { background: #ffc; }
n #summary h2 { font-weight: normal; color: #666; }
n #summary ul#quicklinks { list-style-type: none; margin-bottom: 2em; }
n #summary ul#quicklinks li { float: left; padding: 0 1em; }
n #summary ul#quicklinks>li+li { border-left: 1px #666 solid; }
n #explanation { background:#eee; }
n #template, #template-not-exist { background:#f6f6f6; }
n #template-not-exist ul { margin: 0 0 0 20px; }
n #traceback { background:#eee; }
n #requestinfo { background:#f6f6f6; padding-left:120px; }
n #summary table { border:none; background:transparent; }
n #requestinfo h2, #requestinfo h3 { position:relative; margin-left:-100px; }
n #requestinfo h3 { margin-bottom:-1em; }
n .error { background: #ffc; }
n .specific { color:#cc3300; font-weight:bold; }
n </style>
n <script type=
\"text/javascript
\">
n <!--
n function getElementsByClassName(oElm, strTagName, strClassName){
n
Written by Jonathan Snook, http://www.snook.ca/jon;\\n Add-ons by Robert Nyman, http://www.robertnyman.com\\n var arrElements = (strTagName ==
\"*
\" && document.all)? document.all :
n oElm.getElementsByTagName(strTagName);
n var arrReturnElements = new Array();
n strClassName = strClassName.replace(/

-/g,
\"



-
\");
n var oRegExp = new RegExp?(
\"(
|



s)
\" + strClassName +
\"(



s|$$)
\");
n var oElement;
n for(var i=0; i<arrElements.length; i++){
n oElement = arrElements[i];
n if(oRegExp.test(oElement.className)){
n arrReturnElements.push(oElement);
n }
n }
n return (arrReturnElements)
n }
n function hideAll(elems) {
n for (var e = 0; e < elems.length; e++) {
n elems[e].style.display = 'none';
n }
n }
n window.onload = function() {
n hideAll(getElementsByClassName(document, 'table', 'vars'));
n hideAll(getElementsByClassName(document, 'ol', 'pre-context'));
n hideAll(getElementsByClassName(document, 'ol', 'post-context'));
n }
n function toggle() {
n for (var i = 0; i < arguments.length; i++) {
n var e = document.getElementById(arguments[i]);
n if (e) {
n e.style.display = e.style.display == 'none' ? 'block' : 'none';
n }
n }
n return false;
n }
n function varToggle(link, id) {
n toggle('v' + id);
n var s = link.getElementsByTagName('span')[0];
n var uarr = String.fromCharCode(0x25b6);
n var darr = String.fromCharCode(0x25bc);
n s.innerHTML = s.innerHTML == uarr ? darr : uarr;
n return false;
n }
n
-->
n </script>
n</head>
n<body>
n
n<div id=
\"summary
\">
n <h1>\"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n; _erbout.concat((h exception.class ).to_s); _erbout.concat \" at \"; _erbout.concat((h path ).to_s); _erbout.concat \"</h1>
n <h2>\"\n; _erbout.concat((h exception.message ).to_s); _erbout.concat \"</h2>
n <table><tr>
n <th>Ruby</th>
n <td>
n\"\n\n\n\n; if first = frames.first ; _erbout.concat \"
n <code>\"\n; _erbout.concat((h first.filename ).to_s); _erbout.concat \"</code>: in <code>\"; _erbout.concat((h first.function ).to_s); _erbout.concat \"</code>, line \"; _erbout.concat((h frames.first.lineno ).to_s); _erbout.concat \"
n\"\n; else ; _erbout.concat \"
n unknown location
n\"\n\n; end ; _erbout.concat \"
n </td>
n </tr><tr>
n <th>Web</th>
n <td><code>\"\n\n\n\n; _erbout.concat((h req.request_method ).to_s); _erbout.concat \" \"; _erbout.concat((h(req.host + path)).to_s); _erbout.concat \"</code></td>
n </tr></table>
n
n <h3>Jump to:</h3>
n <ul id=
\"quicklinks
\">
n <li><a href=
\"#get-info
\">GET</a></li>
n <li><a href=
\"#post-info
\">POST</a></li>
n <li><a href=
\"#cookie-info
\">Cookies</a></li>
n <li><a href=
\"#env-info
\">ENV</a></li>
n </ul>
n</div>
n
n<div id=
\"traceback
\">
n <h2>Traceback <span>(innermost first)</span></h2>
n <ul class=
\"traceback
\">
n\"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n; frames.each { |frame| ; _erbout.concat \"
n <li class=
\"frame
\">
n <code>\"\n\n; _erbout.concat((h frame.filename ).to_s); _erbout.concat \"</code>: in <code>\"; _erbout.concat((h frame.function ).to_s); _erbout.concat \"</code>
n
n \"\n\n; if frame.context_line ; _erbout.concat \"
n <div class=
\"context
\" id=
\"c\"\n; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"
\">
n \"\n; if frame.pre_context ; _erbout.concat \"
n <ol start=
\"\"\n; _erbout.concat((h frame.pre_context_lineno+1 ).to_s); _erbout.concat \"
\" class=
\"pre-context
\" id=
\"pre\"; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"
\">
n \"\n; frame.pre_context.each { |line| ; _erbout.concat \"
n <li onclick=
\"toggle('pre\"\n; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"', 'post\"; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"')
\">\"; _erbout.concat((h line ).to_s); _erbout.concat \"</li>
n \"\n; } ; _erbout.concat \"
n </ol>
n \"\n\n; end ; _erbout.concat \"
n
n <ol start=
\"\"\n\n; _erbout.concat((h frame.lineno ).to_s); _erbout.concat \"
\" class=
\"context-line
\">
n <li onclick=
\"toggle('pre\"\n; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"', 'post\"; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"')
\">\"; _erbout.concat((h frame.context_line ).to_s); _erbout.concat \"<span>...</span></li></ol>
n
n \"\n\n; if frame.post_context ; _erbout.concat \"
n <ol start='\"\n; _erbout.concat((h frame.lineno+1 ).to_s); _erbout.concat \"' class=
\"post-context
\" id=
\"post\"; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"
\">
n \"\n; frame.post_context.each { |line| ; _erbout.concat \"
n <li onclick=
\"toggle('pre\"\n; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"', 'post\"; _erbout.concat((h frame.object_id ).to_s); _erbout.concat \"')
\">\"; _erbout.concat((h line ).to_s); _erbout.concat \"</li>
n \"\n; } ; _erbout.concat \"
n </ol>
n \"\n\n; end ; _erbout.concat \"
n </div>
n \"\n\n; end ; _erbout.concat \"
n </li>
n\"\n\n; } ; _erbout.concat \"
n </ul>
n</div>
n
n<div id=
\"requestinfo
\">
n <h2>Request information</h2>
n
n <h3 id=
\"get-info
\">GET</h3>
n \"\n\n\n\n\n\n\n\n; if req.GET and not req.GET.empty? ; _erbout.concat \"
n <table class=
\"req
\">
n <thead>
n <tr>
n <th>Variable</th>
n <th>Value</th>
n </tr>
n </thead>
n <tbody>
n \"\n\n\n\n\n\n\n\n\n; req.GET.sort_by { |k, v| k.to_s }.each { |key, val| ; _erbout.concat \"
n <tr>
n <td>\"\n\n; _erbout.concat((h key ).to_s); _erbout.concat \"</td>
n <td class=
\"code
\"><div>\"\n; _erbout.concat((h val.inspect ).to_s); _erbout.concat \"</div></td>
n </tr>
n \"\n\n; } ; _erbout.concat \"
n </tbody>
n </table>
n \"\n\n\n; else ; _erbout.concat \"
n <p>No GET data.</p>
n \"\n\n; end ; _erbout.concat \"
n
n <h3 id=
\"post-info
\">POST</h3>
n \"\n\n\n; if req.POST and not req.POST.empty? ; _erbout.concat \"
n <table class=
\"req
\">
n <thead>
n <tr>
n <th>Variable</th>
n <th>Value</th>
n </tr>
n </thead>
n <tbody>
n \"\n\n\n\n\n\n\n\n\n; req.POST.sort_by { |k, v| k.to_s }.each { |key, val| ; _erbout.concat \"
n <tr>
n <td>\"\n\n; _erbout.concat((h key ).to_s); _erbout.concat \"</td>
n <td class=
\"code
\"><div>\"\n; _erbout.concat((h val.inspect ).to_s); _erbout.concat \"</div></td>
n </tr>
n \"\n\n; } ; _erbout.concat \"
n </tbody>
n </table>
n \"\n\n\n; else ; _erbout.concat \"
n <p>No POST data.</p>
n \"\n\n; end ; _erbout.concat \"
n
n
n <h3 id=
\"cookie-info
\">COOKIES</h3>
n \"\n\n\n\n; unless req.cookies.empty? ; _erbout.concat \"
n <table class=
\"req
\">
n <thead>
n <tr>
n <th>Variable</th>
n <th>Value</th>
n </tr>
n </thead>
n <tbody>
n \"\n\n\n\n\n\n\n\n\n; req.cookies.each { |key, val| ; _erbout.concat \"
n <tr>
n <td>\"\n\n; _erbout.concat((h key ).to_s); _erbout.concat \"</td>
n <td class=
\"code
\"><div>\"\n; _erbout.concat((h val.inspect ).to_s); _erbout.concat \"</div></td>
n </tr>
n \"\n\n; } ; _erbout.concat \"
n </tbody>
n </table>
n \"\n\n\n; else ; _erbout.concat \"
n <p>No cookie data.</p>
n \"\n\n; end ; _erbout.concat \"
n
n <h3 id=
\"env-info
\">Rack ENV</h3>
n <table class=
\"req
\">
n <thead>
n <tr>
n <th>Variable</th>
n <th>Value</th>
n </tr>
n </thead>
n <tbody>
n \"\n\n\n\n\n\n\n\n\n\n\n; env.sort_by { |k, v| k.to_s }.each { |key, val| ; _erbout.concat \"
n <tr>
n <td>\"\n\n; _erbout.concat((h key ).to_s); _erbout.concat \"</td>
n <td class=
\"code
\"><div>\"\n; _erbout.concat((h val ).to_s); _erbout.concat \"</div></td>
n </tr>
n \"\n\n; } ; _erbout.concat \"
n </tbody>
n </table>
n
n</div>
n
n<div id=
\"explanation
\">
n <p>
n You're seeing this error because you use <code>Rack::ShowExceptions?</code>.
n </p>
n</div>
n
n</body>
n</html>
n\"\n\n\n\n\n\n\n\n\n\n\n\n\n\n; _erbout.force_encoding(ENCODING)", @enc=#<Encoding:US-ASCII>, @filename=nil>>, @logger=#<Logger:0x000000027f6578 @progname=nil, @level=1, @default_formatter=#<Logger::Formatter:0x000000027f6528 @datetime_format=nil>, @formatter=nil, @logdev=#<Logger::LogDevice:0x000000027f6370 @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<IO:<STDOUT>>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x000000027f6280 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x000000027f6078>>>>, @debug_mode=true, @all_resources=@origins=["*", @resources=@path="*", @methods=[:get, :post, :options, @credentials=true, @max_age=1728000, @pattern=/(.*?)$/, @public_resource=true, @headers=:any, @expose=nil>], @public_resources=true>]>, @threaded=nil, @can_persist=true, @idle=false> rack.errors #<IO:<STDERR>> rack.input #<StringIO:0x00000004a88500> rack.multiprocess false rack.multithread false rack.request.cookie_hash {"labwiki.session.4000"=>"BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw\nZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki\nCHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy\nLmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl\ncnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg==\n--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe"} rack.request.cookie_string labwiki.session.4000=BAh7CEkiD3Nlc3Npb25faWQGOgZFRiJFYWZjZDUwMjA0OGJlYzk4N2VkMjEw%0AZGFmOTg4NjhmOTNjMjAyOTRjYTE0OGJhYzVlZTU5YzllNzQxYWZjMzM0ZEki%0ACHNpZAY7AEZJIhVzMTc1NjQyOF8zMzQxMDI3BjsARkkiHHdhcmRlbi51c2Vy%0ALmRlZmF1bHQua2V5BjsAVEkiQ2h0dHBzOi8vcG9ydGFsLmdlbmkubmV0L3Nl%0AcnZlci9zZXJ2ZXIucGhwL2lkcGFnZT91c2VyPWdlbWluaTcwBjsARg%3D%3D%0A--af2af9596b0d7498f812b72f8ea4ab9bb7ac5afe rack.request.query_hash {} rack.request.query_string rack.run_once false rack.session {"session_id"=>"afcd502048bec987ed210daf98868f93c20294ca148bac5ee59c9e741afc334d", "sid"=>"s1756428_3341027", "warden.user.default.key"=>"https://portal.geni.net/server/server.php/idpage?user=gemini70"} rack.session.options {:path=>"/", :domain=>nil, :expire_after=>nil, :secure=>false, :httponly=>true, :defer=>false, :renew=>false, :sidbits=>128, :secure_random=>SecureRandom?, :secret=>"4000", :coder=>#<Rack::Session::Cookie::Base64::Marshal:0x000000029fea00>} rack.session.unpacked_cookie_data {"session_id"=>"afcd502048bec987ed210daf98868f93c20294ca148bac5ee59c9e741afc334d", "sid"=>"s1756428_3341027", "warden.user.default.key"=>"https://portal.geni.net/server/server.php/idpage?user=gemini70"} rack.url_scheme http rack.version [1, 0] warden Warden::Proxy:39071580 @config={:default_scope=>:default, :scope_defaults=>{}, :default_strategies=>{:_all=>[:openid]}, :intercept_401=>true, :failure_app=>AuthFailureApp?}

--

Attachments (1)

labwiki_buffill.log (96.2 KB) - added by dbhat@bbn.com 10 years ago.

Download all attachments as: .zip

Change History (6)

comment:1 Changed 10 years ago by johren@bbn.com

Resolution: fixed
Status: newclosed

Related to the GES communication (DNS) issue. Fixed.

comment:2 Changed 10 years ago by johren@bbn.com

Cc: divyashri.bhat@gmail.com thierry.rakotoarivelo@nicta.com.au added
Milestone: GEC20GEC21
Resolution: fixed
Status: closedreopened
Version: WrapUpSprint1

Reopening because we hit this at the GEC tutorial. It seems that it is not necessarily triggered by one person logging in with many slices, but it is triggered when a few (four or five?) people log in at the same time. I will try to reproduce on emmy9 to start. Divya and Thierry have logs captured from when this was occurring during the tutorial.

Changed 10 years ago by dbhat@bbn.com

Attachment: labwiki_buffill.log added

comment:3 Changed 10 years ago by johren@bbn.com

Owner: changed from jack.hong@nicta.com.au to johren@bbn.com
Status: reopenedassigned

This is being tracked in Labwiki ticket 1792: http://mytestbed.net/issues/1792

Jack has created a fix and Jeanne is trying to test the fix on emmy9.

comment:4 Changed 10 years ago by johren@bbn.com

Version: Sprint1Sprint2

comment:5 Changed 10 years ago by johren@bbn.com

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.