What broke in the last 3 days
50 open production errors in mind-api. Open any card for the full briefing.
NoMethodError
undefined method '[]' for nil "at this link:\n\n#{response['secureShortURL']}" ^^^^^^^^^^^^^^^^^^
User-facing request · 11 people hit it · New this week (4 d old) · Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
User-facing request · 13 people hit it · New this week (1 d old) · Quiet for 40 h
StandardError
[Medplum::Client] GET Practitioner/<secret> failed with HTTP 410: {"resourceType":"OperationOutcome","id":"gone","issue":[{"severity":"error","code":"deleted","details":{"text":"Gone"}}],"extension":[{"url":"https://med…
A staging rake task pages through all Medplum Patients with _offset and hits Medplum's 10,000 offset cap. The 400 is then reported to Bugsnag as a warning by Medplum::Client.
6 people hit it · Happening now · SC-170887 filed
NoMethodError
undefined method '[]' for nil update!(zoom_meeting_id: meeting_data['id'], zoom_care_extender_url: zoom_care_extender_short_url['secureShortURL'], zoom_member_url: zoom_member_short_url['secureShortURL'], ^^^^^^^^^^^^^^…
User-facing request · 24 people hit it · Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
User-facing request · 24 people hit it · Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
User-facing request · 23 people hit it · Quiet for 40 h
ActiveRecord::RecordNotFound
Couldn't find ActiveStorage::Blob with 'id'=3108808
User-facing request · 2 people hit it · New this week (1 d old) · Quiet for 40 h
Oj::ParseError
unexpected character (after ) at line 1, column 1 [parse.c:764]
User-facing request · 3 people hit it · New this week (1 d old) · Quiet for 41 h · SC-171617 filed
NoMethodError
undefined method 'user_id' for nil user_id = audits.reorder(id: :desc).first.user_id ^^^^^^^^
claim_denied_by crashes when a denied billing item has no audit rows, and the code on releases/master still has no guard for that.
User-facing request · 3 people hit it · Quiet for 14 h
Medplum::Labs::UpstreamError
Health Gorilla practitioner sync failed: Medplum Practitioner Practitioner/<secret> missing NPI
A single lab launch failed because the acting practitioner's Medplum profile has no NPI, which the Health Gorilla sync bot needs.
User-facing request · 1 person hit it · New this week (6 d old) · Quiet for 15 h
Faraday::TimeoutError
Net::ReadTimeout with #<TCPSocket:(closed)>
The data-warehouse token request made during SAML login timed out; this is a rare, transient failure of an external service.
User-facing request · 2 people hit it · Quiet for 17 h
Errno::ENOENT
No such file or directory @ rb_sysopen - /app/tmp/CommunicationNote_<name>.pdf
The PDF file was never written to tmp, so reading it back failed; the filename is built from the raw patient name, which is not sanitized, and that is the likeliest cause.
User-facing request · 1 person hit it · New this week (5 d old) · Quiet for 18 h
Medplum::Eprescribe::UpstreamError
ScriptSure patient sync failed: Multiple ScriptSure practices are available for this prescriber; specify an organizationId to choose one. Candidates: Organization/<secret>, Organization/<secret>, Organization/<secret>
The patient sync bot ran without an organizationId, so ScriptSure refused to pick between the prescriber's 3 practices; the service handled it as designed and sent a warning to Bugsnag.
User-facing request · 1 person hit it · New this week (5 d old) · Quiet for 23 h
NoMethodError
undefined method 'new' for nil ActiveStorage.variant_transformer.new(transformations.except(:format)) ^^^^
User-facing request · 2 people hit it · Quiet for 40 h
StandardError
[Medplum::CareExtender] Practitioner update for CareExtender/714 refused by Medplum (HTTP 410)
Medplum answered HTTP 410 (Gone) when we updated the Practitioner linked to CareExtender 1250 on staging. The stored medplum_practitioner_id most likely points to a Practitioner that no longer exists, probably after a staging Medplum reset or cleanup.
New this week (6 d old) · Happening now
Faraday::TimeoutError
Net::ReadTimeout with #<TCPSocket:(closed)>
A login request timed out while waiting for the data warehouse's /oauth/token endpoint to respond; this happened once and the code has no timeout or rescue for it.
User-facing request · 1 person hit it · Quiet for 15 h
ActiveModel::UnknownAttributeError
unknown attribute 'email_type' for MemberEmail. raise UnknownAttributeError.new(self, name) ^^^^^
Updating a member email for a member with no Medplum patient id writes `email_type` straight to the MemberEmail model, which has no such column, so it raises UnknownAttributeError.
User-facing request · 1 person hit it · Quiet for 16 h
NoMethodError
undefined method 'start_date' for nil member_program_statuses.first.start_date ^^^^^^^^^^^
MemberProgram#duration crashes with a nil error when a member program has no status rows, and that breaks the whole member_programs index response.
User-facing request · 1 person hit it · Quiet for 18 h
StandardError
[Medplum::Reader] Patient <secret> not read: HTTP 429
User-facing request · New this week (1 d old) · Quiet for 37 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
User-facing request · 1 person hit it · Quiet for 41 h
NoMethodError
undefined method '[]' for nil update_columns(treatment_plan_doc_url: response['secureShortURL']) ^^^^^^^^^^^^^^^^^^
User-facing request · 1 person hit it · Quiet for 41 h
AbstractController::ActionNotFound
The action 'index' could not be found for HomeController
User-facing request · 1 person hit it · Quiet for 42 h
StandardError
DAP summary webhook for unknown note_id: <secret>
User-facing request · 1 person hit it · Quiet for 43 h
NoMethodError
undefined method 'status_completed?' for nil validate :answered, if: -> { member_assessment.status_completed? } ^^^^^^^^^^^^^^^^^^
User-facing request · 1 person hit it · Quiet for 44 h · SC-150480 filed
ActionController::UnknownHttpMethod
ZCSK, accepted HTTP methods are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK, VERSION-CONTROL, REPORT, CHECKOUT, CHECKIN, UNCHECKOUT, MKWORKSPACE, UPDATE, L…
User-facing request · 1 person hit it · Quiet for 52 h
OAuth2::Error
{"detailedmessage":"You do not have access to this context.","name":"ContextAccessError"}
Athena rejects one or more enabled AthenaContexts with "You do not have access to this context". The job catches the error, logs it and reports it to Bugsnag on purpose, so nothing is crashing.
Happening now · SC-149433 filed
NoMethodError
undefined method 'send_plain_text_email' for class GenericMailer
Queued mail jobs for GenericMailer.send_plain_text_email failed because that action was deliberately removed in a deploy, and no code in the repo enqueues it any more.
New this week (5 d old) · Still arriving
ActiveRecord::StatementInvalid
PG::UndefinedTable: ERROR: missing FROM-clause entry for table "member_content_view_summaries" LINE 1: ...ents" WHERE "contents"."is_independent" = $1 AND (member_con... ^
Members with no programs, tags or care managers get a Content query that never joins member_content_view_summaries, but refresh_member_contents filters and sorts on that table, so Postgres raises.
Still arriving · SC-171159 filed
ActiveRecord::RecordInvalid
Validation failed: Care extender brands must have at least one brand, Roles must include at least one role
db/seeds.rb creates the bootstrap admin CareExtender without brands or roles, which the model now rejects, so the seed fails whenever that admin record is missing.
Still arriving
NoMethodError
undefined method 'program' for nil program_settings = member_program.program.program_settings ^^^^^^^^
The text-message job crashes when a member has no member program, because `assignable_support_request_id_with_class_name` never checks for nil.
Candid::Errors::ClientError
{"errorName":"HttpRequestValidationError","errorInstanceId":"<secret>","content":{"fieldName":"organization_name","humanReadableMessage":"Either organization_name or first + last name must be set"}}
Candid rejected a claim because a provider on it has neither an organization name nor first and last name, and this comes from provider data in Candid.
RuntimeError
Request error (HTTP 422): organization_name: Either organization_name or first + last name must be set [requestId: <secret>]
Candid rejects the claim with a 422 because the billing provider it receives has no organization_name and no first/last name, and our code reports each rejection to Bugsnag as a warning.
OpenSearch::Transport::Transport::Errors::BadRequest
[400] {"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: The number of terms [109372] used in the Terms Query request has exceeded the allowed maximum of [65536]. This maximum can …
1 person hit it · Quiet for 27 h · SC-154178 filed
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
New this week (1 d old) · Quiet for 40 h
NoMethodError
undefined method '[]' for nil @docusign_email_link = url_genertor['secureShortURL'] ^^^^^^^^^^^^^^^^^^
New this week (1 d old) · Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
New this week (1 d old) · Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
New this week (6 d old) · Quiet for 40 h
NoMethodError
undefined method '[]' for nil docusign_email_link = url_genertor['secureShortURL'] ^^^^^^^^^^^^^^^^^^
New this week (6 d old) · Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
New this week (1 d old) · Quiet for 40 h
NoMethodError
undefined method '[]' for nil material_document_link = url_genertor['secureShortURL'] ^^^^^^^^^^^^^^^^^^
New this week (1 d old) · Quiet for 40 h
Google::Apis::RateLimitError
rateLimitExceeded: Rate Limit Exceeded
New this week (6 d old) · Quiet for 44 h · SC-171746 filed
ActiveRecord::RecordInvalid
Validation failed: Mime type is not a valid reference data value of type mime_type, value provided:image/svg+xml
A user sent an SVG file in chat, and the app only accepts the mime types it has seeded, so ChatMessage creation was rejected.
Quiet for 12 h
StandardError
{ "error": { "code": 403, "message": "SenderId mismatch", "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.firebase.fcm.v1.FcmError", "errorCode": "SENDER_ID_MISMATCH" } ] } }
FCM rejected an Android push with SENDER_ID_MISMATCH, which usually means a device token was issued for a different Firebase project than the one used to send it. The code only reports this to Bugsnag as a warning.
Quiet for 12 h
Aws::S3::Errors::NoSuchKey
The specified key does not exist.
The hourly census job listed a file in S3, then found it gone when it tried to read it. This is most likely two runs overlapping, for example around a deploy or restart.
Quiet for 12 h · SC-161969 filed
Twilio::REST::RestError
[HTTP 400] 21211 : Unable to create record Invalid 'To' Phone Number: +1000000XXXX https://www.twilio.com/docs/errors/21211
Twilio rejects the welcome text because the member's phone number (shown as +1000000XXXX) is not a valid number, and nothing checks the number before sending.
Quiet for 13 h
NoMethodError
undefined method 'jid=' for an instance of Medplum::WriteOutboxSweepJob instance.jid = job_hash["jid"] ^^^^^^
Quiet for 31 h
NoMethodError
undefined method '[]' for nil documents_link = url_genertor['secureShortURL'] ^^^^^^^^^^^^^^^^^^
Quiet for 40 h
StandardError
{"message":"Internal Server Error","statusCode":500,"success":false}
Quiet for 40 h
Aws::Sigv4::Errors::MissingCredentialsError
missing credentials, provide credentials with one of the following options: - :access_key_id and :secret_access_key - :credentials - :credentials_provider raise Errors::MissingCredentialsError ^^^^^^^^^^^^^^^^^^^^^^^^^^…
Quiet for 41 h
ArgumentError
wrong number of arguments (given 2, expected 0..1) caller: /app/vendor/bundle/ruby/4.0.0/gems/candidhealth-1.27.0/lib/candid/tasks/v_3/client.rb:112 | raise error_class.new(response.body, code: code) ^^^^
Quiet for 59 h