What broke in the last 24 hours
21 open production errors in mind-api. Open any card for the full briefing.
StandardError
[Medplum::Client] GET Practitioner/<secret> failed with HTTP 410: {"resourceType":"OperationOutcome","id":"gone","issue":[{"severity":"error","code":"deleted","details":{"text":"Gone"}}],"extension":[{"url":"https://med…
A staging rake task pages through all Medplum Patients with _offset and hits Medplum's 10,000 offset cap. The 400 is then reported to Bugsnag as a warning by Medplum::Client.
6 people hit it · Happening now · SC-170887 filed
NoMethodError
undefined method 'user_id' for nil user_id = audits.reorder(id: :desc).first.user_id ^^^^^^^^
claim_denied_by crashes when a denied billing item has no audit rows, and the code on releases/master still has no guard for that.
User-facing request · 3 people hit it · Quiet for 13 h
Medplum::Labs::UpstreamError
Health Gorilla practitioner sync failed: Medplum Practitioner Practitioner/<secret> missing NPI
A single lab launch failed because the acting practitioner's Medplum profile has no NPI, which the Health Gorilla sync bot needs.
User-facing request · 1 person hit it · New this week (6 d old) · Quiet for 15 h
Faraday::TimeoutError
Net::ReadTimeout with #<TCPSocket:(closed)>
The data-warehouse token request made during SAML login timed out; this is a rare, transient failure of an external service.
User-facing request · 2 people hit it · Quiet for 17 h
Errno::ENOENT
No such file or directory @ rb_sysopen - /app/tmp/CommunicationNote_<name>.pdf
The PDF file was never written to tmp, so reading it back failed; the filename is built from the raw patient name, which is not sanitized, and that is the likeliest cause.
User-facing request · 1 person hit it · New this week (5 d old) · Quiet for 17 h
Medplum::Eprescribe::UpstreamError
ScriptSure patient sync failed: Multiple ScriptSure practices are available for this prescriber; specify an organizationId to choose one. Candidates: Organization/<secret>, Organization/<secret>, Organization/<secret>
The patient sync bot ran without an organizationId, so ScriptSure refused to pick between the prescriber's 3 practices; the service handled it as designed and sent a warning to Bugsnag.
User-facing request · 1 person hit it · New this week (5 d old) · Quiet for 23 h
StandardError
[Medplum::CareExtender] Practitioner update for CareExtender/714 refused by Medplum (HTTP 410)
Medplum answered HTTP 410 (Gone) when we updated the Practitioner linked to CareExtender 1250 on staging. The stored medplum_practitioner_id most likely points to a Practitioner that no longer exists, probably after a staging Medplum reset or cleanup.
New this week (6 d old) · Happening now
Faraday::TimeoutError
Net::ReadTimeout with #<TCPSocket:(closed)>
A login request timed out while waiting for the data warehouse's /oauth/token endpoint to respond; this happened once and the code has no timeout or rescue for it.
User-facing request · 1 person hit it · Quiet for 15 h
ActiveModel::UnknownAttributeError
unknown attribute 'email_type' for MemberEmail. raise UnknownAttributeError.new(self, name) ^^^^^
Updating a member email for a member with no Medplum patient id writes `email_type` straight to the MemberEmail model, which has no such column, so it raises UnknownAttributeError.
User-facing request · 1 person hit it · Quiet for 16 h
NoMethodError
undefined method 'start_date' for nil member_program_statuses.first.start_date ^^^^^^^^^^^
MemberProgram#duration crashes with a nil error when a member program has no status rows, and that breaks the whole member_programs index response.
User-facing request · 1 person hit it · Quiet for 18 h
OAuth2::Error
{"detailedmessage":"You do not have access to this context.","name":"ContextAccessError"}
Athena rejects one or more enabled AthenaContexts with "You do not have access to this context". The job catches the error, logs it and reports it to Bugsnag on purpose, so nothing is crashing.
Happening now · SC-149433 filed
ActiveRecord::StatementInvalid
PG::UndefinedTable: ERROR: missing FROM-clause entry for table "member_content_view_summaries" LINE 1: ...ents" WHERE "contents"."is_independent" = $1 AND (member_con... ^
Members with no programs, tags or care managers get a Content query that never joins member_content_view_summaries, but refresh_member_contents filters and sorts on that table, so Postgres raises.
Happening now · SC-171159 filed
NoMethodError
undefined method 'send_plain_text_email' for class GenericMailer
Queued mail jobs for GenericMailer.send_plain_text_email failed because that action was deliberately removed in a deploy, and no code in the repo enqueues it any more.
New this week (5 d old) · Still arriving
ActiveRecord::RecordInvalid
Validation failed: Care extender brands must have at least one brand, Roles must include at least one role
db/seeds.rb creates the bootstrap admin CareExtender without brands or roles, which the model now rejects, so the seed fails whenever that admin record is missing.
Still arriving
NoMethodError
undefined method 'program' for nil program_settings = member_program.program.program_settings ^^^^^^^^
The text-message job crashes when a member has no member program, because `assignable_support_request_id_with_class_name` never checks for nil.
Candid::Errors::ClientError
{"errorName":"HttpRequestValidationError","errorInstanceId":"<secret>","content":{"fieldName":"organization_name","humanReadableMessage":"Either organization_name or first + last name must be set"}}
Candid rejected a claim because a provider on it has neither an organization name nor first and last name, and this comes from provider data in Candid.
RuntimeError
Request error (HTTP 422): organization_name: Either organization_name or first + last name must be set [requestId: <secret>]
Candid rejects the claim with a 422 because the billing provider it receives has no organization_name and no first/last name, and our code reports each rejection to Bugsnag as a warning.
ActiveRecord::RecordInvalid
Validation failed: Mime type is not a valid reference data value of type mime_type, value provided:image/svg+xml
A user sent an SVG file in chat, and the app only accepts the mime types it has seeded, so ChatMessage creation was rejected.
Quiet for 12 h
StandardError
{ "error": { "code": 403, "message": "SenderId mismatch", "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.firebase.fcm.v1.FcmError", "errorCode": "SENDER_ID_MISMATCH" } ] } }
FCM rejected an Android push with SENDER_ID_MISMATCH, which usually means a device token was issued for a different Firebase project than the one used to send it. The code only reports this to Bugsnag as a warning.
Quiet for 12 h
Aws::S3::Errors::NoSuchKey
The specified key does not exist.
The hourly census job listed a file in S3, then found it gone when it tried to read it. This is most likely two runs overlapping, for example around a deploy or restart.
Quiet for 12 h · SC-161969 filed
Twilio::REST::RestError
[HTTP 400] 21211 : Unable to create record Invalid 'To' Phone Number: +1000000XXXX https://www.twilio.com/docs/errors/21211
Twilio rejects the welcome text because the member's phone number (shown as +1000000XXXX) is not a valid number, and nothing checks the number before sending.
Quiet for 13 h